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ABSTRACT 


Theory  of  Stochastic  Optimal  Tracking  Systems 

by 

Vaqar  H.  Syed 

Doctor  of  Philosophy  in  Engineering 
University  of  California,  Irvine,  1977 

Professor  James  S.  Meditch,  Dissertation  Supervisor 
Professor  Allen  R.  Stubberud,  Chairman 


The  object  of  this  dissertation  is  to  study  the 
optimal  tracking  of  signals  modeled  as  stochastic  processes, 
by  linear  plants.  The  signal  available  to  the  plant  is  a 
given  stochastic  process  in  the  presence  of  a white  noise. 
The  criterion  for  optimization  is  the  minimization  of  the 
original  stochastic  process  and  the  plant  output.  The 
study  thus  involves  the  design  of  appropriate  compensators 
to  give  the  systems  the  desired  tracking  properties. 

The  present  theory  of  stochastic  optimal  tracking,  in 
the  mean-square  sense,  only  considers  stationary  systems. 

The  main  thrust  of  this  work  is  to  extend  the  existing 
theory  to  include  nonstationary  systems.  Thus 
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nonstationary  stochastic  processes,  time-varying  plants 
and  sensors,  and  arbitrary  initial  times  arc  admissible 
in  this  work.  Moreover,  due  to  the  nonstationary 
nature  of  the  systems,  state-space  techniques  are  exclu- 
sively used  here.  This  approach  is  a clear  departure  from 
the  frequency  domain  techniques  of  the  present  theory. 

The  systems  in  the  open-loop  as  well  as  the  closed- 
loop  configurations  are  studied.  For  each  case,  the 
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appropriate  compensators  are  designed  both  in  terms  of 
their  impulse  response  functions  and  in  terms  of  their 
state-space  realizations.  Finally,  the  conditions  for 
the  stability  of  the  resulting  systems  are  derived. 
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CHAPTER  1 


INTRODUCTION 


1.1  GENERAL 

The  problem  of  optimal  tracking  of  random  signals  by 
linear  plants  is  a basic  ingredient  of  many  communications 
and  control  system  designs.  A qualitative  statement  of  the 
problem  is  as  follows: 

Given  a dynamic  plant  and  a random  signal  corrupted 
by  an  additive  noise  as  its  input,  how  can  one 
modify  the  plant  dynamics  such  that  the  plant  out- 
put follows  the  input  in  an  optimal  manner? 

Obviously,  this  vague  question  must  be  quantified  to 
obtain  a tractable  design  problem.  To  accomplish  this  end, 
we  make  reasonable  assumptions  about  the  signal,  the  noise, 
and  the  plant,  and  define  a suitable  criterian  for 
"optimality" . 

In  this  dissertation,  we  will  consider  only  linear 
plants.  The  plant  parameters  are  allowed  to  be  time- 
varying,  but  are  required  to  be  continuous  in  the  entire 
interval  of  interest.  The  random  signal  is  assumed  to  be 
a stochastic  process  which  can  be  generated  by  passing 
white  noise  through  a finite-dimensional  linear  system 
with  arbitrary  initial  conditions.  The  parameters  of  the 
system  generating  the  stochastic  process  can  also  vary 
with  time,  but  must  also  be  continuous  throughout  the  time 
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interval  of  interest.  Note  that  the  resulting  process  in 
general  will  be  nonstationary.  All  noise  processes  are 
assumed  to  be  white,  although  colored  noise  processes  can 
be  easily  accommodated  in  this  theory.  The  optimization 
criterion  used  here  is  such  that  the  mean-square  tracking 
error,  defined  as  the  difference  between  the  plant  output 
and  the  original  random  signal  is  minimized.  Finally, 
since  the  modification  of  plant  dynamics  actually  involves 
placing  an  appropriate  compensator  in  the  system,  care  will 
be  taken  to  keep  the  mean-square  value  of  the  compensator 
output  to  within  some  specified  bounds. 

As  pointed  out  above,  the  tracking  problem  requires 
the  design  of  an  appropriate  compensator  to  modify  the 
plant  dynamics.  The  compensator  can  be  placed  in  the 
system  in  either  of  the  two  configurations  shown  in  Figure 
1.1  and  1.2.  The  compensator  as  shown  in  Figure  1.1  will 
be  called  an  "Open-loop  Cascade  Compensator",  and  the  one 
shown  in  Figure  1.2  will  be  termed  a "Closed-loop  Cascade 
Compensator".  The  related  design  problems  will  be  called 
"the  Open-loop  Problem"  and  the  "Closed-loop  Problem", 
respectively . 
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1 . 2 HISTORICAL  BACKGROUND 


A special  case  of  the  above  problem  was  studied  by 
Wiener  in  the  context  of  stochastic  filter  design  [i]. 

His  problem  consisted  of  the  design  of  an  estimator  (filter) 
whose  output  would  optimally  follow  a random  signal 
available  in  the  presence  of  an  additive  white  noise  (see 
Figure  1.3). 

white  noise 

1 

A 
yd 

> Estimate 

Filter 

WIENER  FILTER 
Figure  1.3 

He  modeled  the  random  signal  as  a stationary  stochastic 
process  with  rational  power  density  spectrum  and  sought  the 
transfer  function  of  an  optimal  filter  such  that  the  mean- 
square  error  between  the  filter  output  and  the  given 
random  signal  is  minimized. 

Thus,  if  the  "plant"  in  this  work  is  a scalar  identity 
system  (impulse  response  6(t-T)),  the  random  signal  and  the 
noise  are  stationary,  and  there  is  no  constraint  on  the 
compensator  output,  the  stochastic  optimal  tracking 
problem  reduces  to  the  Wiener  filtering  problem. 
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The  optimal  tracking  problem  as  formulated  here  was 
first  studied  by  Newton  et  al . and  the  results  were 
presented  in  [2] . They  assumed  the  plant  to  be  single- 
input/single-output , linear,  time-invariant,  asymptotically 
stable,  and  characterized  by  a rational  transfer  function. 
The  requirements  for  the  random  signal,  the  noise,  and  the 
optimization  criterian  were  the  same  as  those  of  the 
Wiener  problem.  They  sought  the  design  of  a closed-loop 
cascade  compensator  of  Figure  1.2.  Since,  as  pointed  out 
in  [23],  a direct  formulation  of  the  closed-loop  problem 
was  difficult,  they  reformulated  the  problem  as  an  open- 
loop  problem.  Once  the  transfer  function  of  the  open-loop 
cascade  compensator  was  obtained,  the  conversion  to  the 
closed-loop  design  was  direct  via  transfer  function 
manipulations . 

The  results  of  [2]  were  extended  by  Weston  and 
Bongiorno  [3]  to  the  case  of  multi-input/multi-output 
plants  and  vector  stochastic  processes.  More  recently, 
Youla,  Jabr  and  Bongiorno  in  [4,5]  treated  the  closed- loop 
problem  for  both  single-input/single-output  and  multi- 
input/multi-output systems.  In  addition  to  designing  an 
optimal  closed-loop  compensator,  they  arrived  at  the 
necessary  and  sufficient  conditions  for  its  existence  and 
the  stability  of  the  closed-loop  system. 

It  should  be  noted  that  the  work  cited  above  was 
carried  out  strictly  for  stationary  systems.  That  is,  the 
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plants  were  linear  and  time-invariant,  and  the  signals 
were  stationary  stochastic  processes  over  semi- inf inite 
intervals  (-«,t],  characterized  by  rational  power  density 
spectra.  The  compensator  design  was  therefore  carried  out 
in  the  frequency  domain  employing  Wiener's  frequency 
domain  spectral  factorization  techniques.  In  the  work 
presented  in  this  dissertation,  the  above  assumptions  are 
totally  relaxed.  Thus  the  system  admits  time-varying 
plants,  nonstationary  stochastic  processes,  and  finite 
observation  intervals. 

1.3  CONNECTION  WITH  THE  LQG  TRACKING  PROBLEM 

A particular  stochastic  optimal  tracking  problem  called 
the  "LQG  Tracking  Problem"  has  been  studied  extensively  as 
a special  case  of  the  so  called  LQG  (Linear  Quadratic 
Gaussion)  Regulator  Problem  [lO] . In  the  LQG  Tracking 
Problem,  the  plant  and  the  stochastic  processes  are  allowed 
to  be  nonst- cionary . The  optimization  criterian  is  the  so 
called  "integral  of  the  mean-square  error",  together  with 
a saturation  like  constraint  on  the  "integral  of  the  mean- 
square  control". 

As  shown  in  [10] , by  augmenting  the  plant  dynamics 
with  the  dynamics  of  the  system  generating  the  given 
stochastic  process  (called  the  reference  system) , the  LQG 
Tracking  Problem  can  be  formulated  in  the  context  of  the 
LQG  Regulator  Problem  — the  solution  of  which  is  well 
known  [10] . The  compensator  structure  of  the  LQG  Tracking 
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Problem  is  particularly  interesting,  and  is  shown  in 
Figure  1,4  below. 

In  Figure  1.4  the  state  estimator  estimates  the  state 
of  the  reference  system  generating  the  given  stochastic 
process,  and  the  observer  reconstructs  the  state  of  the 
plant.  The  control  is  simply  a linear  combination  of  the 
states  of  the  reference  system  and  the  plant.  Note  that 
the  above  system  configuration,  together  with  the  feedback 
loop  around  the  plant,  is  inherent  in  the  LQG  Tracking 
Problem, whereas  in  the  Mean-square  Tracking  Problem,  which 
is  the  subject  of  study  here,  the  system  configuration  is 
totally  free. 

1.4  SUMMARY  OF  RESULTS  AND  CONTRIBUTIONS  OF  THIS  RESEARCH 

Both,  the  open- loop  and  the  closed -loop,  problems  are 
treated  in  this  dissertation.  Due  to  the  nonstationary 
nature  of  the  system,  it  proves  natural  that  the  problems 
be  formulated  and  solved  in  the  time  domain.  Since 
extensive  computer  algorithms  are  available  for  state- 
space  related  computations,  the  final  design  of  the  optimal 
compensator  is  given  in  terms  of  its  state-space  realiza- 
tion. 

First,  the  open-loop  problem  is  formulated  in  the 
time  domain.  Variational  techniques  are  then  used  to 
arrive  at  a Wiener-Hopf  type  integral  equation  which  the 
optimal  compensator  must  satisfy.  Wiener's  spectral 
factorization  theory  is  employed  to  solve  the  resulting 
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integral  equation.  New,  special  techniques  to  carry  out 
the  required  factorization  in  the  state  space  are 
developed  in  this  disseration.  These  techniques  enable 
one  to  arrive  quite  naturally  at  the  space  realization  of 
the  optimal  compensator. 

For  the  closed-loop  case,  as  pointed  out  in  [23],  the 
problem  formulation  in  the  time  domain  is  not  so  straight- 
forward. To' accomplish  this  end,  Volterra’s  "Compositional 
Algebra",  which  was  developed  originally  as  an  aid  in  the 
study  of  integral  equations,  is  used.  The  compositional 
algebra  is  applied  here  to  explicitly  formulate  the  closed- 
loop  problem.  Once  again  the  standard  variational  tech- 
niques are  used  to  arrive  at  the  necessary  and  sufficient 
conditions  which  the  optimal  closed-loop  compensator  must 
satisfy.  The  resulting  integral  equation  is  solved  using 
the  same  techniques  as  for  the  open-loop  problem  to  arrive 
at  the  state-space  realization  of  the  closed-loop 
compensator . 

Contributions  of  this  Research 

This  research  addresses  a fundamental  unsolved 
problem  in  systems  theory,  which  is,  the  generalization  of 
the  existing  theory  of  stochastic  optimal  tracking  to 
include  nonstationary  systems.  Thus  a unified  theory  of 
stochastic  optimal  tracking  involving  linear,  time- 
varying  plants  and/or  nonstationary  processes  is  developed. 
Many  of  the  previous  results  therefore  become  special 
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cases  of  the  results  developed  here. 

Another  significant  contribution  of  this  research  is 
the  development  of  general  state-space  spectral  factoriza- 
tion techniques.  The  implicit  close  relationship  between 
the  Riccati  equation  and  the  spectral  factorization  has 
been  known  since  Kalman's  work  [l 2] . However  use  of 
equivalence  transformations  in  state-space  to  accomplish 
spectral  factorization  clearly  and  explicitly  demonstrates 
this  relationship. 

It  is  also  demonstrated  in  this  dissertation  that  the 
optimal  minimum  mean-space  compensator  can  be  separated 
into  a Kalman  State  Estimator  and  a dynamic  system.  The 
previous  work  on  this  problem,  and  the  techniques  used 
fail  to  demonstrate  this  important  property. 

Finally,  Volterra's  compositional  algebra  techniques 
are  applied  here  for  the  first  time  to  time-varying 
feedback  system  optimizations.  These  techniques  may 
prove  to  be  useful  to  other  problems  of  this  nature. 

1.5  ORGANIZATION  OF  THIS  DISSERTATION 

The  next  chapter,  Chapter  2,  deals  with  general 
system  theory  concepts,  mathematical  preliminaries  and 
linear  filtering  theory.  The  topics  presented  are  the 
ones  which  are  used  or  referred  to  in  the  subsequent 
chapters.  In  Chapter  3,  the  open-loop  Problem  is  formulated 
and  solved.  The  optimal  compensator  is  first  solved  for  in 
terms  of  its  impulse  response,  and  then  in  terms  of  an 
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explicit  state-space  realization.  In  Chapter  4,  the 
closed-loop  problem  is  addressed.  Again,  both  the  impulse 
response  and  the  state-space  solutions  of  the  optimal 
compensator  are  given.  Chapter  5 addresses  the  problem  of 
the  open-loop  and  closed-loop  stability  of  the  system,  and 
conditions  are  derived  which  guarantee  the  asymptotic 
stability  of  the  system.  Finally,  in  Chapter  6,  a 
Conclusion  is  presented  and  recommendations  are  made  for 
additional  work  in  this  area. 
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CHAPTER  2 


SYSTEM  THEORY  AND 
MATHEMATICAL  CONCEPTS 

2 . 1 GENERAL 

A complete  understanding  of  the  material  in  this 
dissertation  requires  a knowledge  of  general  system  theory, 
probability  theory,  stochastic  processes,  and  some  real 
analysis.  Such  knowledge  will  generally  be  assumed.  In 
this  chapter,  however,  basic  definitions,  concepts,  and 
notation  fundamental  to  other  chapters  will  be  presented. 
Additional  concepts  will  be  introduced  as  required  in  the 
sequel.  The  primary  references  for  this  chapter  are 
D'Angelo  [8] , Popoulus  [ll] , and  Meditch  [7]. 

2 . 2 LINEAR  SYSTEMS 

The  state-space  description  of  a linear  p-input/q- 
output  time-varying  dynamical  system  is  given  by 

x(t)  = A(t)x(t)  + B(t)u(t) 

y(t)  - C(t)x(t)  + D(t)u(t)  (2.1) 

x(t0)  = xQ 

where  x(t)  is  an  n-vector  of  state  variables,  u(t)  is  a 
p-vector  of  input  variables,  the  control;  and  y(t)  is  a 
q-vector  of  output  variables.  Furthermore,  A(*).  B ( • ) , 

C(*)  and  D(*)  are  matrix  functions  of  the  time  variable  t, 
and  are  of  the  order  nxn,  nxp  , qxn  and  qxp  , respectively.  The 
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dot  denotes  the  time  derivative,  tg  indicates  the  initial 
time  and  Xq  the  system  initial  state. 

The  above  set  of  equations,  for  the  sake  of  brevity, 
can  also  be  written  as  [A,B,C,D;  t^.x^].  If  A(t)  is 
continuous,  and  B(t)  and  u(t)  are  piecewise  continuous  for 
all  time  t,  the  solution  of  the  above  equation  is 


t 

X ( t)  = 4>(t,tQ)x(t0)  +y\(t.T)B(T)u(T)dT 

t0 

for  all  t>tQ,  where  <*>(t,T)  is  the  state  transition  matrix 
of  the  system.  If  the  system  is  initially  in  the  zero 
state,  that  is  x(tQ)=0,  tl 
y(t)  for  t>tQ  is  given  by 


state,  that  is  x(tQ)=0,  the  response  of  the  output  variable 


where 


y(t)  =J  K(t,T)u(T)dT  + D(t)u(t) 
t0 

K(t.r)  = C(t)4»(t,T)B(x) 


The  matrix  K(t,t)  is  called  the  impulse  response  matrix  of 
the  system. 

A linear,  relaxed,  dynamical  system  is  said  to  be 
causal  or  physically  realizable  if 

K(t,x)  = 0 for  all  t<x 
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Equivalence 

Two  systems  are  said  to  be  zero  state  equivalent  if 
both  systems  have  identical  output  when  excited  from  the 
zero  state  with  identical  inputs.  Two  systems  are  zero- 
input  equivalent  if  initial  states  (not  necessarily  equal) 
exist  so  that  both  systems  have  identical  outputs  with  zero 
inputs . 

There  is  a class  of  transformations  that  can  be 
applied  to  the  linear  system  characterized  by  Eq . (2-1), 
that  always  results  in  systems  that  are  zero-state  and 
zero-input  equivalent.  In  particular,  the  equivalence 
transformation  is  defined  by 


x(t)  = T(t)x(t) 

where  T(t)  is  an  nxn  matrix,  nonsingular  and  continuously 
differentiable  on  [tg,t].  Applying  the  equivalence  trans- 
formation to  the  system  of  Eq . (2.1)  results  in 


where 


x(t) 

= A(t)x(t)  + 

B(t)u(t) 

y(t) 

= C(t)x(t)  + 

D( t)u ( t) 

A(t) 

= [T(t)A(t)  i 

- T( t)] T~ 

B(t) 

= T(t)B(t) 

C(t) 

- C(t)T-1 (t) 

D(t) 

- D(t) 

I 

(2.2) 
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The  impulse  response  matrix  of  the  tranformed  system  (2.2) 
is  identical  to  that  of  the  original  system,  whereas  the 
state  transition  matrix  of  the  transformed  system  is 
given  by 

*(t,x)  = T(t)4.(t,x)T'1(T) 

Adjoint  Systems 

A linear  system  S , characterized  by  its  state  transi- 

3 

tion  matrix  $>  (t,i)  and  its  impulse  response  matrix  K ( t , t ) 
a a 

is  the  adjoint  of  a linear  system  S characterized  by  its 
state  transition  matrix  $>(t,i)  and  its  impulse  response 
matrix  K(t,r)  if  and  only  if 

(I)  *a(t,x)  - = *’(T,t) 

o 

and 

(II)  Ka(t,t)  = -K’(T.t) 

for  tg^-rst;  where  prime  denotes  the  transpose.  If  a 
system  satisfies  (II)  only,  it  is  termed  an  input-output 
adjoint  system. 

Theorem  2 . 1 

If  a linear  system  S is  characterized  by 

x(t)  = A(t)x(t)  + B(t)u(t) 
y (t)  = C(t)x(t)  + D(t)u(t) 

then  the  adjoint  S of  S satisfying  properties  (I)  and  (II) 

3 

i 

above  is  given  by 
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xa(t)  = -A’(t)xa(t)  + C,(t)ua(t) 
ya(t)  = +B'(t)xa(t)  + D'(t)ua(t) 

Proof 

Refer  to  [8]  . 

Inverse  Systems 

A system  S”^  is  an  inverse  system  of  a system  S if, 
when  S and  S ^ are  cascaded,  the  output  of  the  combined 
system  is  identical  with  its  input.  Inverse  systems  are 
classified  as  pre-inverse  systems  and  post-inverse 
systems.  A post-inverse  system  for  a p-input/q-output 
system  can  exist  only  if  p>q,  while  a pre-inverse  system 
of  S can  exist  only  if  p<q. 

Theorem  2 . 2 

Consider  a single  input/single-output  system  S defined 

by 

x(t)  = A(t)x(t)  + b(t)u(t) 
y (t)  = c(t)x(t)  + d(t)u(t) 

where  x,  the  state  is  an  n-vector,  and  u and  y are  the 
scalar  input  and  output  respectively.  If  d(t)/0  for  all 
te[tQ,«>),  then  the  following  set  of  equations  represent  an 
inverse  S ^ of  S in  the  sense  that  if  y is  the  response  of 
S to  the  input  u on  [tQ,®)  and  the  initial  state  Xq,  then 
u is  the  response  of  S to  an  input  y on  [tQ,«)  and  the 
same  initial  state.  Thus  S ^ is  given  by 
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z(t)  = [A(t)  - d-1(t)b(t)c(t)] z(t)  + d'1 (t)b(t)y(t) 
u(t)  = -d_1(t)c(t)z(t)  + d_1(t)y(t) 

*<t0)  = xo 

Proof 

See  Silverman  [21] . 

2.3  STOCHASTIC  PROCESSES  AND  LINEAR  FILTERING 

Consider  a continuous-time  measurement  process  model 
of  the  form 

z(t)  = yd(t)  + v(t),  t>tQ  (2.3) 

Here  y^(  •)  is  a Gaussian  distributed  stochastic  process 
signal  vector  and  v(*)  is  a zero  mean  Gaussian  vector  noise 
process  with  convariance 


E[v(t)v' (t)]  = R( t)  6( t- T)  ; t,T>t0 


The  notation  E denotes  ensemble  average  or  expectation,  the 
prime  denotes  matrix  transposition  and  6(*)  is  the  Dirac- 
delta  function.  The  quantity  z(«)  is  called  the  measure- 
ment or  the  observation  process.  It  is  common  to  assume 
that  the  signal  yd(0  is  generated  by  a finite-dimensional 
linear  model 


4»(t)  = A(t)  ip ( t ) + B(t)w(t) 
yd(t)  - C(t)  i|>(t) 


(2. A) 
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for  t>tg.  Here  A(*).  B ( • ) , and  C(-)  are  continuous 
matrix  functions  in  the  time  variable  t,  and  the  process 
w(*)  is  zero-mean  and  Gaussian  distributed  with  covariance 


I 

I I 


E[w(t)w'(T)]  = Q(t)  6(t-x)  ; t,x>t0 

Furthermore,  the  initial  state  ^(t^)  of  the  system  given 
by  Eq  (2.4)  is  a Gaussian  random  variable  with  mean  and 
covariance  Pq,  viz., 

= iJ/q 

E[<P(t0)4>  (tQ)]  = Pq 

Generally,  we  shall  assume  that  v(-)  and  w(»)  are 
uncorrelated;  that  is 

E[v(t)w'(T>]  = 0 (2.5) 

for  all  t and  r. 

When  we  consider  the  case  where  there  is  correlation 
between  x(«)  and  v(*),  we  shall  assume  that 


E[w(t)v'(x)]  = R(t)6(t-T)  (2.6) 

It  should  be  noted  that  the  description  of  w( • ) and  v(*) 
as  white  noise  processes  and  the  model  description  of  the 
form  (2.3)  and  (2.4)  is  by  no  means  rigorous.  A more 
rigorous  representation  is  the  stochastic  differential 
equation  [l5] . However,  the  above  notation  is  more 
familiar  in  linear  systems  theory  and  the  results  obtained 


18 


here  under  this  representation  do  not  differ  from  the 
corresponding  results  if  stochastic  differential  equations 
are  used. 

We  further  assume  that  Q(*)  is  a nonnegative  semi- 
definite  matrix,  and  that  R(*)  is  positive  definite  so  that 
R-^"  exists. 

Let  S(t,r)  be  the  impulse  response  of  a causal, 
minimum-mean-square-error  filter  which  operates  on  z(») 
for  t£tQ  with  as  its  output.  Then 


?d(o  -/ 


S(t,r)z(r)dr 


is  a linear  functional  of  the  measurement  process  z(i), 
tQ<TSt,  that  minimizes 

E[[yd(t)-yd(t>]  [yd(t)-yd(t)]] 

It  is  well-known  that  S(t,r)  satisfies  the  Wiener- 
Hopf  type  equation 


f S(t,r)E[z(r)z(a)]  dx  = E[yd(t)z(a)] 


(2.7) 


for  all  tQ<a<t 


Theorem  2 . 3 


Continuous-time  Kalman-Bucy  Filter.  Given  the  model 
(2.3),  the  filter  S(t,n)  defined  by  Eq . (2.7)  is  represented 
by  the  matrix  differential  equation 

J(t)  = [A(t)  - K(t)C(t)]J(t)  + K(t)z(t)  (2.8) 

yd(t)  = c(tH(t) 

where 

K(t)  = P(t)C(t)R_1(t)  (2.9a) 

in  the  case  where  there  is  no  correlation  between  input 
and  output  noise  processes,  i.e.,  (2.5)  applies,  and 

K(t)  = [P(t)C(t)  + B(t)R(t)]R_1(t)  (2.9b) 

in  the  situation  where  (2.6)  applies.  The  matrix  P(t) 
defined  as  E[[yd-yd] Cyd-yd]'j  is  the  positive  definite 
solution  of  the  matrix  Riccati  differential  equation 

P(t)  = A(t)P(t)  + P ( t) A ' ( t)  - K( t) R( t) K ' ( t)  + B(t)Q(t)B'(t) 

subject  to  the  initial  condition 

P(£q)  t Q ) ^ (t0)]  = Pq 

Proof 

Refer  to  [7,12]. 

Theorem  2 . 4 

(The  Innovations  Theorem.)  Given  (2.3),  the  innova- 
tions process  u(*)  defined  by 
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u (t)  = z(t)  - yd(t) , t* tQ 

is  a zero  mean,  white  Gaussian  noise  process  with  the  same 
covariance  as  v(*  ) . i.e., 

E[u  (t)u'(T  )]  = R(t)6  (t-T  ) , 

Furthermore,  the  processes  z(*)  and  u(*)  are  related  by  a 
causal  linear  operation. 

Refer  to  [l3,14]  for  a proof  and  detailed  discussion 
of  Theorem  2.4. 

Comment 

From  Theorem  2.4  u(*)  and  z(*)  are  related  by  a 

causal  linear  operation.  That  is,  u(*)  can  be  obtained  from 

z(»)  by  passing  z(*)  through  a linear  causal  filter, 

commonly  called  the  "Whitening  filter".  Thus 

+<* 

uC*)  = J Ww(t,r)z(r)dx  (2.10) 

- CC 

where  Ww(t,r)  is  the  impulse  response  of  the  whitening 
filter. 

Conversely,  z(*)  can  be  obtained  from  u(*)  by 
passing  u(*)  through  the  inverse  of  the  filter  Ww(t,r). 
Denoting  the  inverse  of  W (t,r)  by  W ”^(t,r)  in  the  sense 

w w 

that 


The  inverse  here  is  defined  in  the  sense  of  Section  2.2. 
The  existence  of  the  inverse  of  Ww(t,x)  is  discussed  in 
Kailath  [13]. 
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-t-OC 

y*Ww(t,n)W"1(n,r)dn  - J W"1 (t . n)Ww(n . r)dn = In 


where  In  is  an  identity  matrix  of  appropriate  dimensions, 


we  have 


>• 


r)  u(r)dx 


(2.11) 
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CHAPTER  3 


THE  OPEN-LOOP  PROBLEM 

3 . 1 GENERAL 

In  this  chapter  the  open-loop  optimal  tracking 
problem  is  formulated  and  solved.  Since  the  systems  under 
study  are  nonstationary,  it  seems  natural  that  we  use 
time-domain  techniques  for  the  analysis  and  design.  The 
final  design  of  the  open-loop  cascade  compensator  is 
presented  in  terms  of  its  state-space  representations.  The 
main  results  of  this  chapter  are  presented  in  the  form  of 
Theorems  3.1  and  3.5.  The  organization  of  the  chapter  is 
as  follows. 

Section  3.2  is  devoted  to  the  state-space  formulation 
of  the  problem.  In  Section  3.3,  a necessary  and  sufficient 
condition  which  the  optimal  open-loop  cascade  compensator 
must  satisfy  is  derived.  This  condition  turns  out  to  be 
a type  of  nonstationary  Wiener-Hopf  integral  equation. 

The  integral  equation  of  Section  3.3  is  then  solved  in 
Section  3.4  to  arrive  at  both  the  impulse  response  and  the 
state-space  representation  of  the  open-loop  cascade 
compensator.  In  Section  3.4,  we  also  develop  the  state- 
space  spectral  factorization  techniques  and  illustrate  them 
via  examples.  In  Section  3.5,  a brief  discussion  and 
critique  of  the  results  is  presented. 
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3.2  PROBLEM  STATEMENT 

Consider  a linear,  time-varying,  single-input/single 
output  plant  DSl  (Figure  3.1)  defined  by 


x(t)  = 

F(t)x(t)  + g(t)u(t) 

(3.1a) 

DSl: 

y(t)  = 

h(t)x(t) 

(3.1b) 

x(tQ) 

= 0 

(3.1c) 

where  x(t)  is  an  n-vector,  the  state;  u is  the  control 
input  and  y is  the  system  output,  both  scalars,  and  F(*). 
g(»)  and  h(»)  are  real,  continuous  matrix  functions  of 
time  t which  are  nxn,  nxl , and  Ixn,  respectively.  The  dot 
here  denotes  the  time  derivative  and  t^  is  the  initial 
time . 

Let  yd(t)  be  the  reference  signal  which  the  output  y(t) 
of  the  plant  DSl  is  required  to  track  optimally.  We  assume 
that  the  signal  y^C*)  is  generated  by  a finite-dimensional, 
linear  model  of  the  form 

iKt)  = A( t)  t|>( t)  + b(t)w(t) 

DS2;  (3.2a) 

yd(t)  * c(t)iKt) 

for  titg.  Here,  i p is  an  m-vector,  the  state;  w(«)  and 
yd(*)  are  scalars,  and  A(*).  b(*),  and  c(*)  are  continuous 
matrix  functions  of  t,  which  are  mxm,  mxl,  and  lxm, 
respectively.  Furthermore,  the  process  (w(t) , t^tg}  is  a 
zero  mean,  Gaussian,  white  noise  process  with  covariance 
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e[w(L)w(t)]  = Q(t)<5  (t-t  ) 


(3.3) 


and  the  initial  state  ^(t^)  of  the  system  given  by 

Eq.  (3.2)  is  a Gaussian  distributed  random  variable  with 

mean  ij>q  and  covariance  Pq  , 

E ( tQ)]  = iJiQ  (3.4a) 

EO(t0H’(t0)]  = PQ  (3.4b) 

The  system  defined  by  Eq.  (3.2)  will  be  termed  the 
re'c*irence  system  DS2. 

The  signal  available  for  input  to  the  plant  DS1  is 
modeled  as 

z(t)  = yd(t)  + v(t)  (3 . 5) 

where  {v(t) , t>tQ}  is  a zero  mean,  Gaussian,  white  noise 
process  with  variance 

E[v(t)  v(t)]  = R(t)  6 (t-t ) , t>tQ  (3.6) 

We  assume  that  Q(t)  and  R(t)  are  both  continuous,  that  Q(t) 
is  nonnegative,  and  that  R(t)  is  positive  definite  such 
that  R ^(t)  exists. 

The  tracking  error  ye(t)  is  defined  by 

ye(t)  - y(t)  - yd(t)  (3.7) 

As  a measure  of  the  system  performance,  we  choose  a 
suitably  weighted  sum  of  the  mean-square  tracking  error 


25 


yg  and  the  mean-square  control  effort  u.  Denoting  the 
index  of  system  performance  (the  cost  functional)  by  L, 
we  thus  have 

L = E[y2(t)]  + k(t)E[u2(t)]  (3.8) 

where  k(t)  >0  is  a Lagrange  multiplier.  A suitable  value  of 
k is  chosen  in  the  last  stages  of  the  design  process  to 
satisfy  a saturation-like  constraint  on  u,  of  the  form 

E[u2(t)]_<0 

where  0>O.  Our  problem  can  now  be  stated  as  follows: 

Given  the  plant  DSl,  the  reference  system  DS2,  the 
relevent  noise  statistics  and  the  constraints  on  u,  design 
and  realize  in  state-space  an  optimal  open-loop  cascade 
compensator  such  that  the  cost  functional  L given  by 
Eq.  (3.8)  is  minimized. 

3.3  DERIVATION  OF  THE  WIENER-1I0PF  INTEGRAL  EQUATION 
A block  diagram  of  the  system  with  which  we  are 
concerned  in  this  chapter  is  shown  below  in  Figure  3.1. 

In  the  diagram,  W(t,r)  and  Wc(t,r)  are  the  impulse  response 
functions  of  the  plant  and  the  optimal  compensator, 
respectively. 

To  keep  track  of  variables,  as  will  become  apparent 
subsequently,  we  shall  use  u instead  of  u in  Eq . (3.8), 
where  u is  a linear  functional  of  u.  That  is 
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-f-cr 

u = J W(t,r)u(r)dr , 


t>t0. 


Later,  we  can  always  choose  W(t,x)  to  be  the  identity 
system  (W(t,r)  = 6(t-r))  such  that  u=u.  Hence,  in  view 
of  Figure  3.1,  a modified  form  of  Eq . (3.8)  is 

Li  = El>d(t>  " 2yd(t>y<t)  + y2(t>]  + k(t)E[u2(t)]  (3.9) 

It  is  clear  that  u(t)  and  y(t)  can  be  written  as 

+<r  +ot 

u(t)  = dp  y*ds  W(t,p)Wc(p,s)z(s)  (3.10) 


-cc  -cc 


and 

+®  +oc 

y(t)  = / dp/  ds  W(t,p)Wc(p,s)z(s)  (3.11) 

- cc  - cc 

respectively,  where  from  causality,  tQ<s<p,  tQ<p<t,  and 
titQ.  Substituting  from  Eqs.  (3.10)  and  (3.11)  into  Eq . 
(3.9)  , we  get 


Lx  - E 


+«  +« 


yd(t)  - 2yd(t)  Jdq J dr  W( t , q)W£ (q ,r) z(r) 


-a  - oc 


I hi 


■ cc  - cc 


ds  W(t,p)Wc(p,s)z(s)  | x 
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+2  -h? 


dr  W(L ,q)Wc(q ,r)z(r) ^ 


+«  +JT 


+ k!/dp/  ds  W(t,p)Wc(p,s)z(s) j x 


-a  -a 


— a -a 


dr  W(t,q)Wc(q,r)z(r) 


(3.12) 


Equation  (3.12)  can  be  rewritten  as 


+«  -foe 


Li  = E yd(t)  - 2yd(t)  /uq/  dr  W(t,q)Wc(q,r)z(r) 


+ OC  +OC  +OC  *4"  of 

J'd p y ds y dq  f dr 

- oc  - oc  - oc  - oc 


W(t.p)Wc(p,s)W(t,q) 


Wc(q,r)z(s)z(r) 


+a  +ct  +a  +<* 


k J dp y* ds J' dq  J dr  W(t ,p)W£(p , s)W(t , q) 


-a  — cc  — oc  -a 


Wc(q,r)z(s)z(r) 


(3.13) 


We  now  assume  the  existence  of  a physically  realizable 
compensator  WcQ(t,s)  that  minimizes  Eq . (3.9).  Next,  we 

proceed  to  use  a well  known  variational  technique  to 

\ 

arrive  at  W (t,s).  We  thus  let 
co 
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(3.1 A) 


Wc(t.s)  = Wco(t,s)  + eWe(t,s) 

where  We(t,s)  is  the  variation  of  Wc(t,s).  Substituting 
Eq.  (3.14)  into  Eq . (3.13),  we  have 


Lx  - E 


+<r  4-« 


yj(t)  ' 2yd(t)y*dq/*dr  W(t,q)[wco(q.r)  + eWe(q,r)jZ(r) 


- cc  -a 


-fee  -fac  "fa  -fa 


.1 


\fdP  j 135  h h W(t,p)W(t,q  )|  wco(P' s)wco(^  *r) 


•a  - a - a - a 


+ eWco(P . s)We(q , r) 


o ) 

+ eW  (p,s)W(q,r)  + e W(p , s)  W(q , r) z (s) z (r) 

c CO  l 


+0C  4-«  +oc  -fee 


h/dp/ds/dq/ dr  W(t,p)W(t,q)  Jwco(p,s)Wco(q,r.) 


- oc  -a  - ac  -a 


+ eWco(p,s)We(q,r) 


2 ) 

+ eW  (p , s)W  (q , r)  + e W(p , s)  W(q , r) z (s) z (r) 
C CO  \ 


(3.13) 


A necessary  condition  for  to  be  a minimum  is 

3L, 


3e 


- 0 


e = 0 


Carrying  out  the  required  differentiation  and 

\ 

recognizing  that  p is  interchangeable  with  q,  and  that  s 
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is  interchangeable  with  r,  we  obtain 


3Ll 
— 3e* 


= E 


e = 0 


+ “ -f  oc 


■2yd(t) J dq J dr  W(t ,q)W(q ,r)z(r) 


-CX  — CC 


-fa  -fa  -fa  -fa 

+ 2 J dp  J dsj  dqj dr  W(t,p)W(t,q) 


- a -a  -a  - cc 


wco(p' s)We(q,r)z(s)z(r) 


-fa  -fa  -fa  -fa 


2k J"  dp J " ds J"  dq  J * dr  W(t,p)W(t,q] 


— a — a -a  -a 


Wco(p,s)We(q,r)z(s)z(r) 


= 0 (3.16) 


If  we  interchange  the  order  of  the  integral  and 
expectation  operators  in  Eq . (3.16)  and  use  the  fact  that 
We(q,r)  = 0 for  r<tQ  and  that  W^fq.r)  is  arbitrary  for 
tQ<r<qstQ,  we  are  led  to  the  following  result. 

Theorem  3 . 1 

For  a physically  realizable  (causal)  compensator 
to  minimize  , it  is  necessary  and  sufficient  that  it 
satisfy 


+ <x  -foe  -fee 


dr  W(t,r)W(t,p)Wco(p,s)E[z(s)z(r)] 


+“  +“  +* 


k/dp/ds y*dr  W(t,r)W(t,p)Wco(p,s)E[z(s)z(r)] 
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+« 

=y*  dr  W(t  pr)E[yd(t)z(r)] 

- oc 

for  all  t>tg,  t q < s < p , tg<p<t,  and  tg<r<t.  (3.17) 

Proof 

Necessity  is  immediate  from  above.  Sufficiency 

follows  from  consideration  of  the  second  variation. 

3.4  SOLUTION  OF  THE  INTEGRAL  EQUATION 

Solution  of  the  Wiener-Hopf  type  integral  equation  of 

Section  3.3  for  a physically  realizable  Wco(t,r)  is 

developed  in  a number  of  steps. 

First,  the  Innovations  Theorem,  Theorem  2.4  and  the 

whitening  filter  given  by  Eqs.  (2.10)  and  (2.11)  are  used 

to  convert  the  Wiener-Hopf  type  integral  equation  involving 

covariances  into  a Wiener-Hopf  type  integral  equation 

involving  operators  only.  This  conversion  process  is 

illustrated  in  Lemmas  3.1,  3.2  and  Theorem  3.2.  Second, 

the  resulting  integral  equation  which  now  involves  adjoint- 

operators,  is  solved  for  a physically  realizable  operator 

representing  the  optimal  compensator.  Wiener's  spectral 

factorization  techniques  are  used  during  this  step,  and 

the  optimal  compensator  arrived  at  is  in  terms  of  its 

impulse  response  Wco(t,r).  Third,  an  explicit  state-space 

realization  of  W (t,r)  is  obtained.  This  realization 
co 

process  is  developed  in  Theorems  3.3  and  3.4. 
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Lemma  3 . 1 


{E[yd(t)u(a)]R'1(a)}  * Ww(t.a)  = S(t,a)  (3.18) 

where  u(*),  W (•,•)  and  S(*,»)  are  respectively,  the 
w 

innovations  process,  the  whitening  filter  and  the 
Kalman-Bucy  filter  associated  with  the  process  z(-),  and 
the  asterisk  denotes  the  convolution  of  linear  operators. 
Proof 

Let  yd(t)  be  the  filtered  estimate  of  yd(t) . Thus 

"4~oc 

yd(t)  -/  S (t , r) z (r)dr , tgSr^t  (3.19) 


where  S(t,r)  satisfies  Eq . (2.7).  Since  z ( •)  and  u(0 
are  informationally  equivalent  to  each  other,  yd(t)  is  also 
given  by 


yd(c)  -/ 


S(t  ,r)  u(r)dr  , tQ<;r<:t 


(3.20) 


where  S(t,r)  satisfies  the  integral  equation 


/ 


S(t,r)E[u(r)u(a)]  dr  = E[y  , (t)  u(a)]  , tnsa<;t  (3.21) 


From  the  innovations  theorem, 

E[u(r)u(a)]  = E[v(r)v(a)]  = R(r)6(r-a)  (3.22) 
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Substituting  Eq . (3.20)  into  Eq . (3.19)  and  carrying  out 
the  integration  with  respect  to  r,  we  have 

S (t , a) R(a)  = E[yd(t)u(a)] 

S(t,a)  = E[y  ,(t)u(a)]R_1 (a) 


or 


(3.23) 


Recall  from  Chapter  2,  Eq . (2.11),  that 


z(t) 


= / W'w1(t,r)u(r 


) dr , tQsrs t 


(3 . 24) 


If  we  substitute  Eq . (3.24)  into  Eq . (3.19)  and  compare 
the  result  with  Eq . (3.20),  we  note  that 

S(t,r)  * - S(t,r) 


or 


S(t  ,r)  = S(t,r)  * Ww(t,r) 


(3.25) 


Thus,  from  Eqs.  (3.23)  and  (3.25),  we  have 

S(t,r)  = ,(E[yd(t)u(r)]R'1(r)}  * Ww(t,r) 


Q.E.D. 


Lemma  3 . 2 


The  integral  equation  (3.17)  and  the  following 
integral  equation  are  equivalent: 
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•fa 


■fa  -fee 


fir  W(t.r)j/dP/d.  W(t,p)Wco(p,s) [Ww  1 (s ,r)R(r)J  | 

- or  - a - a 

+<*  -fa  -fa 

kjdr  W(t  ,r)  | Jdp  yds  W(t,p)Wco(p,s)[Ww"1(s,r)R(r)]J 


- OC  - OC 


-foe 

y*dr  W(t,r)E[yd(t)u(r)] 


(3.26) 


for  t>p>s>tQ,  tg<r<t. 

Proof 

Substituting  Eq . (3 . 24)  into  Eq . (3.17),  we  obtain 

-fa  -fa  -fa 

/dp/ds J dr  W(t,r)W(t,p)Wco(p,s)x 


- a - oc  - oc 


/■></ d"w; 


1(s,C)u(C)u(n)Ww1(r,n) 


■ cc  - OC 


rfa  -fa  -fa 

+ kjdpj'dsj'dr  W(t  ,r)W(t  ,p)WcQ(p  , s)x 


— a — CX  - CC 


[/;/: 

- oc  - a 


dTiW“1(s>Ou(fJu(r,)vr1(r,n) 


+a  +a 

=y*drW(t,r)Ej  /*dn  yd (t)  u(n) 1 (r , n)  | 
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Next,  we  interchange  the  expectation  and  integral 
operators,  substitute  Eq . (3.22)  into  the  above  equation 
and  carry  out  the  integration  of  the  resulting  delta 
function  with  respect  to  C to  obtain 


+o  +<r  40c  +o 

/dp/ds/dr/‘ 

_cc  _ oc  _ oc  _ oc 


d'lW(t,r)W(t,p)Wco(p,s)Ww1(s,n)R(n)W^1(r,n) 


4.0c  +<r  40c  40 


k/dp/ds/dr/ dnW(t,r)W(t,p)Wco(p,s)W"1(s,n)R(n)\r1(r,n) 


.a  .oc  .oc  .a 


+«  +° 


= ^dr J*  dnW(t,r)E[y<J(t)u(n)]W'1  (r  ,n) 


(3.27) 


. oc  . a 


Equation  (3.27)  can  be  rewritten  as 


-|-ac  -f-oc  -f-ar 


Ji dnW^^r.n)  j /dr f d?  f dsW(t  ,r)  W(t  ,p)  WCQ(P , s)W^(s , n)R(n) 

cr  ' *'nr  ^ rr  •'or 


. oc  _ a;  . ac 


4-a  4.0c  4« 


k f dr  f dp  f dsW(t,r)W(t,p)Wco(p,s)Ww1  (s.n)R(n) 

.CC  „ (i  . CC 


/w<t  ,r)E[yd(t)u(n)] J = 0 


Note  that  (r,n)  in  the  above  equation  is  arbitrary  in 
the  sense  that  the  equation  holds  true  for  any  W *(r,n) 
over  the  interval  tQSn<u<t.  Obviously  then,  the  above 
equation  reduces  to 
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■ 


1 

f 


I 


+<r  +oc  +oc 


/dr/dP/ 

- a - oc  -a 

+oc  +oc  4*cr 


ds  W(t,r)W(t,p)Wto(p,s)V;w  ( s , r ) R ( r ) 


,-l 


,r)W(t  ,p)VJco(p,s)Ww  (s  ,r)R(r) 


+ k /dr / dp / ds  W(t,i 

- cc  - cc  - cc 
+<x 

= /dr  W(t,r)E[yd(t)u(r)]  (3.28) 

• CC 

Equation  (3.26)  follows  immediately  from  Eq.(3.28). 


Q.E.D. 

Theorem  3 . 2 

The  integral  equation  (3.28)  and  the  following 
operator  equation  are  equivalent: 

Wa  * W * W + kWa  * W * W = Wa  * S (3.29) 

co  — — co 

where  S is  the  Kalman-Bucy  filter  associated  with  the 
process  z(*).  and  is  given  by  Eqs.  (2.8)  and  (2.9). 

Proof 

The  bracketed  portion  of  the  first;  term  on  the  left- 
hand  side  (LHS)  of  Eq.  (3.26)  is  a convolution  of  the 
linear  operators  W(t,a),  W (t,a)  and  {Ww^ (t , a) R(a) } . We 
denote  the  resulting  operator  by  W^(t.a),  i.e., 

W1(t.a)  = W(t.a)  * Wco(t,a)  * (W^1(t.a)R(a)} 

I 

and  write  the  above  term  in  Eq.  (3.29)  as 
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. A __ 


J da  W(t,a)W1(t,a) 


Now  let  Wa(t,a)  be  the  adjoint  of  the  system  W(t,a). 

Since  W(t,a)  = Wa(a,t),  the  above  expression  can  be  written 
as  the  convolution  of  two  linear  operators  — W (a,t)  which 
is  physically  unrealizable  (noncausal) , and  W^(t,a)  which 
is  physically  realizable  (causal).  Hence,  the  firsL  term 
of  the  LHS  of  Eq . (3.29)  can  be  written  as 

wa  * W * W * W-1R 
CO  w 


Using  the  same  argument  for  the  rest  of  the  terms  of 

Eq . (3.29),  this  equation  becomes 

Wa  * W * W * W-1R  + kWa  * W * W * W-1R  = Wa(a,t) 
co  w — — co  w 

* (E[ytj(t)  u(a)]} 

or 

wa  * W * W * W"1  + kWa  * W * W * W"1  = wa 
co  w — — co  w 

* Efyd(t)u(a)]R-1 

or 

Wa  * W * W + kWa  * W * W - Wa(a , t) 
co  — — co 

* {E[yd(t)u(a)]R"1(a)  ) * Ww(t,a) 


If  we  substitute  the  results  of  Lemma  3.1  into  the 
above  equation,  we  obtain  immediately  the  result 

W*  * W * W + kWa  * W * W = Wa  * S (3.29) 

CO  — — CO 
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for  t*tQ. 

Q.E.D. 


Equation  (3.29)  holds  only  for  t>t^  and  involves 
linear  operators  and  their  adjoints.  It  can  therefore  be 
solved  using  Wiener's  spectral  factorization  techniques 
[1,2].  The  solution  is  developed  in  Eqs.  (3.31)  through 
(3.35)  below.  Before  we  proceed,  we  substitute  for  W an 
identity  system  with  <5(t-T)  as  its  impulse  response.  This 
step  is  in  line  with  the  original  formulation  of  the 
problem.  Equation  (3.29)  thus  reduces  to 


Let 


(Wa  * W + k)*  Wco  = Wa  * S (3.30) 


Wa  * W + k = Ma  * M 


(3.31) 


3 

where  M is  the  physically  nonrealizable  (noncausal) 
adjoint  of  some  operator  M.  Thus 

Ma  * M * W = Wa  * S (3.32) 

co 

or 

M * Wco  = [M3]"1  * Wa  * s (3.33) 

where  [Ma]  ^ is  the  inverse  of  the  system  Ma  in  the  sense 
of  Theorem  2.2. 

Next,  the  right  hand  side  of  Eq . (3.33)  is  written 

3 

as  a "sum"  of  two  systems  described  by  the  operators  L 
and  L2 , where  .L2  is  causal  and  L is  noncausal.  That  is 
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(3.34) 


[M3]"1  * Wa  * S = La  + L 


Thus 


M * W = L0 
co  2 


or 


= M*1  * l9 
co  2 


(3.35) 


Equation  (3.35)  gives  the  impulse  response  of  the 
open- loop  cascade  compensator  Wco  in  terms  of  the 
parameters  of  the  systems  DSl,  DS2,  the  noise  processes 
w,  v,  and  the  Lagrange  multiplier  k(t),  all  imbedded  into 
M ^ and  L2 . The  next  step  is  to  lay  bare  the  structure 
of  M 1 and  L2  in  terms  of  the  system  parameters,  and 
obtain  a realization  of  WcQ  in  the  more  useful  state -space 
form.  To  accomplish  this,  we  carry  out  in  the  sequel,  the 
mathematical  operations  described  by  Eqs.  (3.31)  through 
(3.35)  directly  in  state-space.  To  curb  the  proliferation 
of  symbols,  we  denote  by  u-^  and  y^  the  scalar  input  and 
output,  respectively,  of  any  given  dynamic  system. 
Furthermore,  we  use  the  symbol  "« — >"  to  denote  the 
equivalence  between  the  state-space  and  the  impulse 
response  representations  of  systems. 

Theorem  3 . 3 

Let  (F,g,h;tQ,0)  be  a realization  of  W,  i.e., 

W <— * (F,g,h;t0,0) 
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Furthermore,  let 


Wa  * W + k = Ma  * M 


where  Wa  is  the  noncausal  adjoint  of  W,  Ma  is  the  noncausal 
adjoint  of  some  operator  M,  and  k>0.  Then  (F , g,h  , tg  , 0) 
is  a realization  of  M,  where 


£ = /k 

and 

- * /lb"r 

Here  T is  a symmetric,  nonnegative  definite  nxn  matrix 
satisfying  the  Riccati  Equation 

T = - Tgg'T  + TF  + F'T  + h’h  (3.36) 

subject  to  the  initial  condition 


T<C0>  - °nxn 

Proof 

From  Theorem  2.2. 

Wa  <— > (-F ' ,h*  ,-g'  ,t0,0) 

Denoting  the  states  of  W and  Wa  by  x and  x^  respectively, 
we  have 
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X 

F : 0 

i 

X 

+ 

g 1 

X 

a 

h'h  ; -F' 

X 

a 

0 

. 

u, 


vr  * w + k < — > 


*i 


= [o 


x 

X 


+ kux  <337> 


' x<t0)  ’ 

' 0 ' 

1 

= 

— 

. xa(t0>. 

0 

We  let 


Therefore 


M < — > (f 


Ma  <— » (-F ' ,h’ ,£;tn,0) 


If  we  denote  by  s and  s the  state  vectors  of  M and 
j _ —a 

M , respectively,  we  can  write 


-a 


F 

h'h 


0 

■F' 


s 

-a 


+ 


£ 

h’£ 


u. 


Ma  * M < — > / yx  - [lh  ! -£'J 


-a 


+ S. u, 


(3.38) 


'£<y  ‘ 

ss 

1 

i o 

1 

. £a(t0>  . 

. 0 J 
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Equation  (3.38)  obviously  has  a slightly  different  form 
than  that  of  Eq . (3.37).  To  find  F,£,h,  and  £,  we  trans- 
form Eq . (3.37)  into  the  same  form  as  Eq . (3.38).  The 
following  equivalence  transformation  is  used  for  this 
purpose , 


1 

r i 

o 

nxn  i 

l 

nxn 

1 

T ( t) 

I 

L v ' nxn  , 

nxn 

so  that 


*"1 


(t) 


I 

0 

-T(t) 

I 

1 J 

and  T(t) 


o 

0 

[ T(t>  ; 

1 

1 o 

i 

From  Section  2.2,  we  thus  have 


• _ - 
X 

F 

0 

X 

g 

Tg 

. 

-T  + TF 

h'h  + F'T 

-F1 

X 

-a 

(Wa  * w + k)<— yL  - [g'T  ; -g'j 


x 

-a 


+ ku, 


x(tQ) 

f.°l 

. Sa<tO) . 

1 

1 o 

1 

(3.39) 


A3 


comparing  Eqs.  (3.38)  and  0.39),  wo  note  that 


F 

= F;  £.  = g;  i = k 

Tg  = h'/k 

(3.40) 

g'T  = h/k 

(3.41) 

h'h 

= -T+TF+h'h+F'T 

(3.42) 

Equations  (3.40),  (3 

k>0  imply  that  T is 

and  that 

.41)  together  with  the  requirement  that. 

symmetric  and  nonnegative  definite, 

h - i g'T 

(3.43) 

Furthermore  Eqs.  (3.42)  and  (3.43),  and  the 

conditions  of  W imply  that  the  nxn  matrix  T 

ini tial 

satisfies 

i—*|^ 

u 

• H 

Tgg'T  + TF  + h'h  + F'T 

(3.44) 

T(t0)  = 0 

Q 

. E . D . 

Hence  M and  Ma 

are  given  by 

M < — > | 

s = Fs  3 gu^ 

y\  = a g'T£  + /k  ui 

(3.4b) 

s(tQ)  = 0 
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1 

/k 


(3.46) 


s = -F's  + 
—a  -a 


Tgu , 


Mc'  < — > { yl  = -g'sa  + /k  ul 


sa(t0)  = 0 


Example  3.1 

The  following  example  illustrates  the  decomposition 
technique  of  Theorem  3.3.  Let  W be  given  by 

x = -x  + u 
y = z 
x(tQ)  = 0 


Let  k=l.  Therefore  T is  given  by 

T = -T2-2T  + 1 
T(tQ)  - 0 

Solving  the  above  Ricatti  equation  we  obtain 
T = ( /2  - 1 ) + (l-/2)e"2y/2(t"tO) 


For  tg — ->  -i  we  have 

T = /2-1 

r 

Thus  a state  space  representation  for  M in  the  steady 
state  is 

s - -s  + u^ 
y = (/2-l)s  + 
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As  a check,  we  Laplace  transform  the  above  equation 
to  obtain  the  transfer  function  M(s)  of  M.  Thus 


M(s) 


s±/2 

s+1 


From  classical  spectral  factorization  techniques,  we  have 


Wa  * W + k 


1 


-s+1 


-s 


s+1 


+ 1 


+ 1 


+2 
+ 1 


-S+/2  s-t/2 

-s+1  x s+1 


or 


M(s) 


s+/2 

s+1 


Theorem  3 . A 

Assume  that  we  arc  given  (a)  the  system  * S'JJ  * S^, 
where  and  are  adjoints  of  some  causal  systems  Sj  and 
S2,  and  is  a causal  system,  (b)  the  following  realiza- 
tions for  , S2  and 


sl«- 

(Flt 

«1» 

V 

V 

t0* 

nxn 

nxl 

lxn 

lxl 

S 2 < — 

(F2 , 

g2- 

hg  i 

t0' 

0) 

nxn 

nxl 

lxn 

S3 

->  (lr3, 

g3* 

h3’ 

L0’ 

0) 

mxm 

mxl 

lxm 

A6 


and  (c)  the  decomposition  rule 


where  is  the  adjoint  of  some  causal  system  and  P2 
is  causal.  Then,  P2  has  the  realization 

P2  <— * (F3,  e3,  Y;  t0,  0) 
mxm  mxl 

where 

Y = + 8{U2 

Ui  = UlF3  + F*U1  + h2h3 

W « 0 

U2  = U2F3  + 1'lU2  + hlB2Ul 
U2(tQ)  = 0 


Alternatively,  Y is  given  by 


nxm 

— TT  ~ 

r f’ 

2 

nxn 

0 

nxn 

U + 

2 nxm 

uf3  + 

h 2 h 3 

U2 

u 

2nxm 

h^g’ 

F1 

(2 nxm) (mxm) 

0 

. nxm  . 

_ nxn 

nxn 

(2nx2n) 


U(tQ)  « 0 


Y 

i 


(3.47) 


47 


Proof 


Let  f>la*  ^2a  and 
and  S^,  respectively. 

tion 


cl  T 

^ be  the  state  vectors  of  S j , SV, 
Then  * S'")  * lias  the  realizn- 


h 

F3 

0 

• 

0 

ha 

= 

*l2^3 

-F2 

0 

ha 

. - 

0 

• 

-h^2 

y 1 

= 

[«•  i 

'*lg2 

-gi] 

u 


1 


0 


(3.48) 


To  decompose  the  above  realization  into  P'j  and  P.^,  v:e  make 
use  of  the  following  equivalence  transformation  on  Eq . 
(3.48), 


i 

0 

0 

(mxm) 

(mxn) 

(mxn) 

U1 

I 

0 

(nxm) 

(nxn) 

(nxn) 

U2 

0 

I 

. (nxm) 

(nxn) 

. (nxn) 

It  is  easy  to  verify  that 
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The  ?2  part  of  the  decomposition  can  be  identified  by 
inspection.  Thus 

1^3  = F3^3  + S3U1 

, 

yl  = ^1P’2U1  + glU2^3 

where  U-^  and  are  given  by 


and 


U1  = U1F3  + F2U1  + h2h3  = 0 


U,(t0)  = 0 


U2  = U2F3  + K;u2  + h1’g’ui 


Defining  U = 


u2(t0)  = 0 


, and  augmenting  the  differential 


equations  in  and  U2 , we  obtain  the  equation  (3.47) 


Q.E.D. 


Example  3.2 

The  above  decomposition  technique  is  illustrated  by 
the  following  example.  Let  Sc^,  and  by  given  by 

S*  *— » (2 , 1 , -1 , 1 ; tQ , 0) 

S*  (1 , 1 , ■ 1 ; tQ , 0) 

S3  * * (-2.1.1; tQ.0) 
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is  given  by 


such  that 


rt  1 

’-2 

0 

X 

a 

= 

1 

1 

z 

_ — a_ 

0 

-1 

yl 

= 

[ 0 

1 

r$(t 

0>1 

‘ 0 " 

xa<tO) 

0 

0 

0 * 

<-=► 

1 

’ 1 • 

0 

X 

+ 

0 

a 

2 

z 

0 

a 

ft 


From  Theorem  3.4  if  S'J  * S‘2  * 
is  given  by 


a = -2a  + 


P*  + P2,  then  P2 


yl  = ^U1  + U2^ - 

where 

Ux  = -31^  + 1 
Ul(t0)  = 0 

and 

(l2  « -4U2  + UL 
u2(t0)  - 0 
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Solving  for  and  , we  obt  ain 


and 


ux(t) 


1 

3 


I c-3(t-t0) 
3 


U2(t)  - f2 


1 e-4(t-t0) . 


1 

3 


To  find  the  steady  state  values  of  and  , we  let 
tQ-»--«  and  obtain 


U 


1 


1 

3 


12 


Therefore,  in  the  steady  state 

a = -2a  + u, 


( ^ 

p2<->| 


or 


yl  12 


P (s) 

2KS)  s+2 


As  a check,  we  compute  V,^  below  via  classical  factor! 
techniques.  Thus 


* S*  * S3 


Therefore 

> P2(s)  = s+2 
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nation 


where 


or 


Lemma  3 . 3 

The  L2  part  of  the  decomposition  piven  by  Lq . (3.34) 
has  the  state  space  representation 


(a  = (A-Kc)a  + Ku^ 

yl  = /k  s'U’i  + u2]3.  (3.^9) 

( a(tQ)  = 0 

where  and  U2  are  obtained  from 

UX  = F’U1  + U^A-Kc)  + h'c  (3.50a) 

U^tQ)  = 0 

U2  = (F'  " A TKS’)U2  + U2(A-Kc)  - Tp.g'Uj  (3.50b) 

u2(t0)  - 0 


Proof 

The  proof  follows  immediately  from  Theorem  3.4. 

, Q.E.D. 

The  next  theorem  is  the  key  result  of  this  chapter. 
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Theorem  3 . 5 


The  optimal  compensator  W = M ^ * I,2  where  M and  I 
are  defined  by  Eqs.  (3.31)  and  (3.36),  respectively,  has  the 
following  realization 


a 

A-Kc 

0 

1 

I \Zi 
\ 

K 

b . 

i 88 'Y 

r-k  88'T. 

b . 

+ 

0 

u 

= 1 

8'Y 

- ¥ 8’tJ 

a 

b 

Here  K and  T are  defined  by  Eqs.  (2.9)  and  (3.66) 
respectively;  and  Y is  an  nxm  matrix  given  by 


Y = Ux  + U2 

where 

Ux  = F' Ul  + UX(A-Kc)  + h' c 
U1(t0)  = 0 

u2  = (F'  - ^ Tgg’)U2  + U2(A-Kc)  - Tgg ' UL 
u2(t0)  = 0 

Alternatively,  Y is  given  by 


1 

r 17* 

F ! 

0 

'_V 

[A-Kc] ) 

h ' c 

. U2  • 

.-Tgg’  i I 

"~A  T88 ' . 

■ U2  - 

+ 

. U2  . 

0 

Y 

= r i 

i i 1 

_”l_ 

Lnxn 

; nxn  J 

. U2  - 
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Proof 


The  proof  follows  immediately  from  Theorem 
3.3,  and  by  augmenting  the  state  vectors  of  M ^ 
Theorem  3.5  thus  gives  an  explicit  state  space 
of  the  optimal  compensator  W . 


3.4,  Lemma 
and  I>2 . 
real ization 


Q.E.D. 


3.5  DISCUSSION  OF  RESULTS 

The  following  comments  about  the  design  of  the 

l 

optimal  compensator  ^CQ  are  in  order.  First,  from  F.q . 

(3.35)  the  optimal  compensator  is  a cascaded  combination 
of  dynamic  systems  L2  and  M ^ . It  is  obvious  from  | 

Eq.  (3.49)  that  L2  can  be  viewed  as  a Kalman-Bucy  filter 
A(t,i)  for  the  system  DS2  with  the  output  of  the  filter 
multiplied  by  ^ Y.  In  block  diagram  form,  the  cascade 
compensator  is  shown  in  Figure  (3.2)  below.  Thus  the 
separation  property  of  the  minimum  mean-square  compensator 
in  the  sense  of  the  LQG  problem  becomes  immediately  clear. 
Second,  from  the  structure  of  the  compensator,  it  is  readily 
apparent  that  the  stability  of  the  Kalman-Bucv  filter  plays 
a major  role  in  the  stability  of  the  compensator  and  the 
overall  system.  This  in  turn  is  tied  to  the  solution  of  the 
Riccati  equation  for  the  filter  error  covariance  P(t) . 
Furthermore,  stability  of  the  Riccati  equation  (3.44)  for  T 
and  the  differential  equation  (3.47)  for  U play  the  key 
roles  in  the  stability  of  the  system.  More  on  this  topic 
will  be  presented  in  Chapter  5.  Third,  the  presence  of  the 
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Lagrange  multiplier  k,  which  is  always  positive,  assures 
the  existence  of  proper  inverses  [Ma]  * and  M ^ . The 
compensator  is  therefore  also  always  proper.  . Finally,  the 
compensator  equations  contain  k as  a parameter.  The 
choice  of  k is  left  to  the  discretion  of  the  system 
designer  who  chooses  that  value  for  k which  satisfies  the 
constraint 

E[u2(t)]<B 


Note:  S(t,r)  = c(t)A(t, r) 


OPTIMAL  COMPENSATOR  STRUCTURE 


Figure  3.2 


CHAPTER  4 


THE  CLOSED  LOOP  PROBLEM 

4.1  GENERAL 

In  Chapter  4,  the  optimal  tracking  problem  in  the 
closed-loop  configuration  of  Fig.  1.1  is  dealt  with. 

Once  again,  due  to  the  nonstationary  nature  of  the  system 
under  study,  we  formulate  and  solve  the  problem  in  the  time 
domain . 

> 

It  should  be  recalled  [2-5]  that  for  the  optimal 
tracking  problems  of  the  class  studied  here,  the  starting 
point  is  the  formulation  of  a suitable  performance  index 
(PI)  in  terms  of  the  system  parameters.  In  the  open- loop 
case  [2,3,22],  such  a formulation  presents  no  difficulty. 
However,  the  case  of  the  closed- loop  problem  is  entirely 
different.  For  example,  from  Fig.  1.2,  a performance  index 
for  the  closed-loop  tracking  problem  is 

L = E'*-yd(t:)  “ y<^2! 

- E Jy^Ct)  - 2yd(t)y(t)  + y2(t)j  (4.1) 

where  E is  the  expectation  operator.  For  simplicity,  let 
us  assume  that  the  feedback  sensor  has  unity  gain  and  no 
memory.  Let  us  proceed  to  express  y ( L ) in  terms  of  the 
system  parameters.  Thus 

y(t)  - /w(t.s)j/Gc(s,p)U(p)  - y(p)]dpjds 
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or 


+0C 

y(t)  + J\i(t,s)J‘cc(s  ,p)y(p)dpds 


= /w(t  , s)  (s , p) ” (p) dp 


(Is 


(A.  2) 


It  is  immediately  obvious  from  Eq . (A. 2)  that  in  order  Lo 
obtain  y(t)  for  further  manipulations  as  part  of  Eq . (4 . I ) , 
one  needs  to  solve  a double-integral  equation  in  v(t)  — 
a not  so  easy  task!  As  mentioned  above,  no  such  difficulty 
arises  in  the  open-loop  case.  This  is  precisely  why  the 
open- loop  approach  has  been  used  so  extensively  in  the  past 
as  an  intermediate  step  towards  solving  the  closed- loop 
tracking  problem. 

We  overcome  the  above-mentioned  difficulties  associated 
with  the  closed-loop  tracking  problem  by  employing  the 
techniques  of  the  "Algebra  of  Compositions  of  Functionals", 
first  developed  by  Evans  and  Vol  terra  [ 1 8 , 1. 9 J in  the  course 
of  their  studies  of  integral  equations.  The  Algebra  of 
Compositions  is  used  here  to  explicitly  formulate  the 
system  performance  index  to  be  minimized  in  terms  of  the 
system  parameters.  Variational  techniques  are  then  used  to 
derive  a condition  which  the  optimal  compensator  must 
satisfy.  This  condition  turns  out  to  be  an  integral 
equation  similar  to  the  one  derived  for  the  open- loop 
problem  of  the  previous  chapter. 
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The  organization  of  this  chapter  is  as  follows.  The 
next  section,  Section  A. 2 is  devoted  to  the  state-space 
formulation  of  the  closed-loop  tracking  problem,  with  which 
we  are  concerned.  Section  A. 3 deals  with  the  "Algebra  of 
Composition"  and  its  application  to  system  theory.  In 
Section  A. A,  an  integral  equation  which  the  optimal  closed- 
loop  cascade  compensator  must  satisfy  is  derived.  In 
Section  A. 5,  the  integral  equation  is  solved  and  a state 
space  realization  of  the  optimal  compensator  is  given. 
Finally,  in  Section  A. 6,  a brief  summary  of  the  results  of 
this  chapter  and  a conclusion  are  presented. 

A. 2 PROBLEM  FORMULATION 

Consider  a linear,  time-varying,  single-input/single- 
output plant  DS1,  a reference  system  DS2  and  a feedback 
sensing  system  DS3  interconnected  in  the  configuration  of 
Fig.  4.1.  The  defining  equations  for  the  plant  DS1  and  the 
reference  system  DS2  are  the  same  as  in  Section  3.2,  Chapter 
3,  and  are  repeated  below  for  convenience. 

x(t)  = F(t)x(t)  + git)u(t) 
y (t)  = h(t)x(t)  (A. 3) 

x(tQ)  = 0 

>l>( t)  = A(t)'Ht)  -I  b ( t ) w ( t ) 
yd(t)  = c(tH(t) 

E ( t q ) ] = 'I'g 

EO(t0H  <t0>]  = r0 
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k 

» 


where  x,4>  ,F,g,h,u,A,b,c  ,w  are  exactly  as  defined  previously 
in  Section  3.2. 

The  input  signal  to  the  tracking  system  is  modeled  as 

z(t)  = yd(t)  + v(t)  (4 . 5) 

where  (v(t) , t^tg}  is  a zero-mean,  gaussian,  white  noise 
process  with  positive-definite  variance  R given  by 

E [v(t) v(t )]  = R(t)o(t-i),  T:r>t0 

It  is  assumed  that  the  variance  Q of  w defined  by 

E[w(  t)w(x )]  = Q(t)  6 (t-i ) , T.tMq 

is  nonnegative  definite. 

The  system  output  y is  sensed  by  the  feedback  sensor 
DS3  which  is  governed  by  the  equation 

|a(t)  = ft(t)a(t)  + £(t)y(t) 

yQ(t)  = y(t)a(t)  (4.8) 

a(t0)  = 0 

for  titg.  Here,  a is  an  H-vector,  the  state  of  the  feed- 
back sensor,  y^  is  a scalar,  the  feedback  signal,  and  U, 
and  y are  continuous  matrix  functions  of  the  time,  which 
are  ixl,  £.xl  and  lx£  respectively.  The  tracking  error 
yc(t)  is  defined  as 

yc(t)  - yd(t>  - y<c) 

For  the  system  performance  index  L,  we  choose  a suitably 

I 

weighted  sum  of  the  mean- square  tracking  error  and  the 


(4.6) 


(4.7) 
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mean-square  control  u.  Thus 


L = E[y2(t)]  + k(t)E[u?(t)]  (4.9) 

where  k>0  is  a Lagrange  multiplier  whose  value  is  chosen 
in  the  final  stage  of  the  design  process  to  satisfy  a 
saturation- 1 ike  constraint  on  u of  the  form 

E[u2(t)]iB>0  (4.10) 


The  closed-loop  tracking  problem  can  now  be  stated  as 

follows:  Given  the  plants  DSl,  the  reference  system  DSP, 

the  feedback  sensor  DS3,  the  relevant  noise  statistics  and 

the  constraint  on  u,  design  .an  optimal  compensator  in  the 

closed-loop  cascaded  configuration  of  Fig.  4.1  such  that 

the  cost  fuctional  L given  by  Eq . (4.9)  is  minimized. 

4.3  MATHEMATICAL  PRELIMINARIES,  ALGEBRA  OF  COMPOSITION  OF 
FUNCTIONALS  [ 1 8 , 1 9 J 

Given  two  functions  f(y»0  and  g ( y , O of  two  variables 
Y and  £ , the  function  h(y,£,)  defined  by 


My.O 


/ 


f (y.a)s(cx, 


Od 


is  called  product  by  composition  of  two  functions  f and  g 
and  is  denoted  here  by 


If 


S - ? 

?g  = 


* 

g 


then  f and  g are  said  to  be  permutable.  The  operation  of 
composition  is  evidently  associative  and  distributive,  i.e., 
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and 


(tg)fi  = *<$> 


•V  JUJU  *'»A 

l(g+h)  = fg  + 

where  "+"  denotes  ordinary  addition.  Powers  of  composition 
for  positive  integral  exponents  are  defined  by 

. p.  fn-lj  , fn  . ffn-1 


» 


Power  with  exponent  zero  is  defined  such  that. for  any  f 
and  g 


and 


f° 


*o 

g 


Symbolically,  then  we  can  define  a "unity"  function  1 such 
that 

t *o  '£o 
1 = g = f 

Therefore  for  n functions  * *^n  Y an£*  5 

following  series  is  well  defined. 


jbn 


Fff  f f ) = t + ic  fa^?a?--fan  + k . . fb 

ntrt2-  V 1 + klrl  t2  n K2rl  r2  n 


+ 


Here  k^s  are  constants  and  n-js  and  h^s  are  integers.  The 
series  is  called  a functional  F of  f^,f2,*,,fn. 

Fractional  powers  of  composition  of  functionals  are 

* 

defined  in  the  following  manner.  If  for  a given  function 
h(y,0.  there  exists  another  function  g(y,£,)  such  that 
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*n  t 
g = h 

where  n is  a positive  integer,  then  we  define 

S1/n  - I 

If  ft*5  = for  some  positive  integers  p and  q,  then  we 


define 


fi  = Sq/P  and  t = Sb/fJ 


? . 


We  say  that  — is  a "Fraction  of  Composition  with  numerator 

g ? f 

^ Vr  V'iV  Vf  1 1 1 2 

r and  denominator  g if  fg  = I.  Two  fraction  — — and  — — 

*1  «2 

are  equal  if  ?jg2  = 

The  following  relatioiiships  are  self  evident 


* ** 

_g_  _ _g|^  h v g 

* — **  • ‘Vn  ~~  1 • -V  f ~ 8 

* 5,.  ho  f 


f " R 


Negative  exponents  are  defined  by 


JL  *£-l 

g* 


or  by 


* 

8 


*-l* 
f a 


It  can  be  shown  that  [ 1 8 ] from  any  analytic  function 

Z(z^ , z2 , . . . zn) , regular  in  the  region  around  z^=z2= ■ • • • zn~° . 

we  can  obtain  a corresponding  functional  F( f ^ , f 2 • • • • fn) • 
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Moreover,  if  Z(z,  ,z0....z  ) is  an  infinite  series  which  is 
12  n 

convergent  when  the  module  z are  sufficiently  small,  then 
the  corresponding  series  F(f ^ , f^ . . . . fn)  is  always  convergent 
whatever  fr's  may  be,  provided  they  are  limited.  That  is, 
for  all  fr's  such  that 

| fr(y  ,0\<a 

Furthermore,  the  algebra  for  the  manipulations  of  the 
functionals  and  their  composi tions  i s exact ly  analogous  to 
ordinary  algebra. 

Integrals  and  derivatives  of  functionals  and  their 
compositions  are  defined  in  exactly  the  same  fashion  as 
ordinary  integrals  and  derivatives.  Thus 

dP 

^ (f  i * f 2 » “~^n)  a new  function  by  composition  of 

* 

F. 

Linear  Systems 

The  compositional  algebra  presented  above  is  directly 
applicable  to  linear  systems.  Thus  two  linear  systems  with 
impulse  response  functions  f(t,x)  and  g(t,i),  when 
cascaded,  are  equivalent  to  a system  given  by  the  impulse 
response 

h(t,x)  = /f(t.o)g(o.,)do 

- a 

Or,  alternately,  we  can  write 
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Thus  signals  in  a particular  part  of  an  interconnected 
linear  system  can  be  represented  as  functionals  and  their 
compositions,  operating  on  signals  existing  in  some  other 
part  of  the  system.  The  fact  that  these  functionals  and 
their  compositions  can  be  manipulated  just  like  functions 
in  ordinary  algebra  and  calculus,  is  of  great  value  in 
optimization  problems. 

4.4  THE  INTEGRAL  EQUATION 

In  this  section,  we  derive  a necessary  and  sufficient 
condition  which  the  closed-loop  optimal  cascade  compensator 
must  satisfy. 

In  order  to  keep  track  of  the  variables,  as  it  was 
done  in  the  last  chapter,  here  too  we  choose  to  constrain  a 
functional  u of  u rather  than  u.  Thus  the  revised  cost 
functional  L’  is 

L’  = E[y^  - 2ydy  t y2]  + kE[u2]  (4.11) 

where 

-roc 

U = /w(t,T)u(T)dT , t>tQ 


From  Fig.  4.1 

kt 

e = z - FWGe 

or 

(if  + Fifth  e = z 

or 

e,  - (f  + fifth”  Lz 


I 


Thus 


and 


u = 


[WG(i  + FWG)_1Jz 

(4.12) 

[&G(f  + 1H)G)-1]z 

(4.13) 

For  convenience,  in  what  follows,  we  omit  the  "asterisk1 
Substituting  Eqs.  (4.12)  and  (4.13)  into  F.q . (4.11),  we 
obtain 

i 

■l-l  J 


L'  = E 


yd"2yd|[WG(1+FWG)  H 

2 

+ |[WG(l+FWG)_1]z j + kJ[WG(l+FWG)_1]zj 
Applying  the  calculus  of  variations,  we  let 


G = G + eG 
c e 


where  G is  the  value  of  G which  makes  L'  stationary,  and 
c 

eG  is  the  variation  of  G . Thus 
e c 

L'  - E yd-2ydj[w[G+eGe](l+FW[Gc+eGe])“1]zJ 

+ J [w  [Gc+eGe  ] ( 1+FW  [Gc+eGe  ] ) ' 1 ] z j 2 


+ k/[y[G  c+eGe](l+FW[Gc+eGe])"1]; 


The  stationary  value  of  g£  is  next  found  by  setting 


(A.  14) 


dL’ 

de 


c=0 


0 
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Differentiating  the  right-hand-side  of  Eq . (4.14)  term 
by  term  with  respect  to  e , we  have 
First  Term 

k - 0 

Second  Term 


'2yd  | [HtCc+eGe] <l+FW[Cc+eCe] )_1]z  J 
-2yd|K  ^a+Fwtv^eh'1 

+ He  w[Gc+eGe]x(1+™Gc)'1]!!! 


e=0 


= E 


= E 

-2y  1 
2yd| 

r-WG  ( 1+FWG  )'2FWG  +WG  (1+FWG 

L c c ' e e c'  J 

■( 

= E 

-2y  1 
d( 

-WG  FWG  (1+FWG  )“2+WG  (1+FWG  )_1 
c e o g c 

H 

Third  Term 


dE 


[w  [Gc+eGe] ( 1+FW  [Gc+eGe] ) " l] 


nz! 


=0 


= 2E 


[wGc(l+FWGc)"1]zj 


HE  |[w[Cc+eGe](l+F«[G<;+oGe])-1]: 


'e=0 


2E 


j[wGc(l+FWGc)-1]zjx 
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j [-WGcFWGc  ( l+FWGc)  ' l+WGg  ( l+FWGc  ) " 1 ] J z 


Fourth  Term 


k j[w[Gc+eGe](1+FH[Gc+eGe])'1]Zj 

J =( 

= 2k  E j[wGc(l+FWGc)_1jzJx 

■h  j[«[Gc+eGe](1+™[Gc+eGe))'1]z|  . 


= 2k  E 


-WG  FWG 
— c e 


E j[wGc(l+FWGc)-1]zJx 
( l+FWGc ) ' 2+WGe ( l+FWGc ) ' z J 


We  post-multiply  each  of  the  above  terms  by  the  operator 
2 

(l+FWGc)  , gather  all  four  terms  and  obtain 
E [-2yd  J-WCcFWGe+WGe(l+FWGc)j  z] 

+ 2E  J[wGc(l+FWGc)~1]z|  j[-WGcFWGe+WGe(l+FWGc)]  zj 

+ 2kE  jlwG  (1+FWG  ) “11  zj  jf-WG  FWG  IWG  (I  t FWG  )_1lz.! 
|L-  Cv  c'  J ) I L - C e e e ' J ) 


J 


Post-multiplying  each  term  of  the  above  equation  by  the 


operator  (l+FWGc)  and  simplifying,  we  obtain 


ZE  J[l+FWGc]yd|  ][WGc]z 


+ 2E  [ [WGjzj  j[WGe]zj 
+ 2kE  j[Wc]zj  j[WGe]zj  =0 


(A.  15) 


Since  Ge  = 0 for  tctp  and  arbitrary  for  t>tQ,  Eq.  (A. 15) 
can  be  rewritten  as 


E { [WGc]  z | j [W]z  j + kE  j [WGC]  z[  j [w]z 


I [ W ] z 


for  t*tQ. 


It  should  be  noted  that 


[WGc]z  = W(t,T2)Gc([2,i1)z(T1) 

-or  — a 

+«  +oc 

[WGc]z  = /dx2  /*dTl  W(tll2)fic(T2*Tl)z(Tl) 

— oc  - ac 

+« 

[w]z  - /do  W(t ,o)z(o) 


do  VJ(t  ,o)z(o) 


(A. 16) 
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■fa 

[W]z  = fio  ^ W(t,o)z(o) 

-a 

-fa  -fa  4-oc 

[FWGc]yd  = yj  r3  f dr  l L • 7 3)  W(T  3 - 1 2^Cc  2 ’ T 1 ^ yd  1 ^ 


— a -cr  - ac 


where  t>r  2*t  co  ’ anc*  t^o>tQ.  Substituting  the  integral 
representation  for  the  operator  representation  in  Eq  . (6.16), 
and  interchanging  the  order  of  integral  operators  with  the 
expectation  operator,  we  obtain  the  following  integral 
equation . 

>L“  r+a 

j di  2 Jdjl  J d°  w(c  »T2^Ge^  2 ,T  i^W^t  -u)F'Cz(t  f)  z(fJ)] 

• a - a - a 

J-*  J-*  -+a 

+ k /dt  2 y ix  LJ  cJa  W(t  ,t2)Gc(t2  .T  i>W(t  ,0)E[z(ti)z(o)] 

- oc  - a:  - oc 

-fa 

= VJ(  t , a)  E [yd  ( t)  z (a  )] 

- a 

+cc  -fa  -fee  -fa 

+ F(t  ,t3)W(t3  , t2)Gc(t2  ’ T l>W(t  *°) 


■ OC  - oc 


ECyjCT^zCo)]  (4.17) 


for  t^i2M:2>T^>  t^,  and  t>o>tQ. 


The  key  result  of  this  section  is  given  by  Theorem 
4.1  below. 
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Theorem  4 . 1 


The  necessary  and  sufficient  condition  for  the 
physically  realizable  closed- loop  cascade  compensator  Gcq 
to  minimize  L'  is  that  it  satisfy  the  integral  equation 
(4.17). 

Proof 

The  necessary  part  of  the  proof  is  immediate  from  the 
above  derivation.  The  sufficiency  follows  from  considera- 
tion of  the  second  variation. 

Comment 

In  Eq . (4.17),  if  we  set  F(t,i)?0  (which  is  equivalent 
to  breaking  the  feedback  loop),  the  equation  reduces  to  the 
integral  equation  (3.17)  for  the  open-loop  problem.  This 
is,  of  course,  what  one  should  expect. 

4.5  THE  OPTIMAL  COMPENSATOR 

The  optimal  compensator  Gc(t,i)  is  obtained  by  solving 
the  above  integral  equation  in  the  time  domain.  The 
solution  of  Eq . (4.17)  follows  along  the  lines  of  the 
solution  of  Eq . (3.17).  It  should  be  noted  that  the  first 
three  terms  of  Eq . (4.17)  are  identical  with  the  three 
terms  of  Eq . (3.17).  Therefore,  the  results  of  Lemma  3.2, 
and  Theorems  3.1  and  3.2  are  directly  applicable  here. 

The  following  theorem  summarizes  the  steps  necessary  to 
convert  Eq . (4.17)  involving  covariances  into  an  equation 
involving  operators. 
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Theorem  4 . 2 

The  integral  equation  (4.17)  anti  the  following 
operator  equation  are  equivalent 


Wa  * W * G + k Wa  * W * G 
e - — e 

= Wa  * S + Sa  * S * F * W * G (4.18) 

c 

3 3 

for  tktQ,  where  W and  W are,  respectively,  the  noncausal 
adjoints  of  W and  W;  and  S is  the  Kalman-Bucy  filler 
associated  with  the  process  z(-),  and  is  given  by  Eq . (2.8). 
Proof 

The  proof  follows  immediately  if  we  substitute  for 
z(t)  in  Eq.  (4.17),  the  integral 


/* 

/ w(t ,T)u(x)dT , t<T<tQ 


where  u(*)  is  the  innovation  process  associated  with  z(*), 
and  use  the  arguments  developed  in  Lemmas  3.1,  3.2  and 
Theorem  3.2. 

Q.E.D. 

Since  Eq . (4.18)  holds  for  only  t>tg,  and  involves 
linear  operators  and  their  adjoints,  it  can  be  solved  via 
Wiener's  spectral  factorization  techniques.  Once  again, 
as  was  done  in  the  last  chapter,  we  set  W=6(t-x)  and  obtain 
the  following  equation 
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or 


(Wa  * W + k)  * Gc  = Wa  * S[l  + F * W * Gc] 


(Wa  * W + k)  * G *[l+F*W*G]"1=  Wa 
' c c 

Let 

Wa  * W 4-  k = Ma  * M 

where  Ma  is  the  noncausal  adjoint  of  M.  Thus 

M * G *[1  + F*W*g]’1-  [M3]"1  * Wa  ■ 
c,  c 

where  [Ma]-^  is  the  inverse  of  Ma  in  the  sense  of 

2.2. 

Furthermore,  let 

[M3]"1  * Wa  * S = La  + L2 

where  L is  a causal  operator  and  L2  is  noncausal. 

M * G [l  + F * W * G ]_1  = L0 
c c L 

or 

G = M"1  * L9  * [l  + F * W * G ] 
c z c 

or 

G - M"1  *L*F*W*G  = M-1  * L. 
c 2 co  2 

or 

(1  - M-1  * L * F * W)  * G„rt  = M'1  * L 
' 2 'co 

or 

Gc  = [l  - M"1  * L2*  F * W]"1  * M"1  * L2 


* S 

Theorem 

Then 


(4.19) 


p 


Equation  (4.19)  gives  the  impulse  response  of  the  closed- 
loop  optimal  compensator  in  terms  of  the  operators 
and  W.  Once  again  if  we  set  F(t,T)=0,  Eq . (4.19)  reduces 
to  Eq . (3.35),  the  equation  for  the  open-loop  compensator 
of  the  last  chapter.  The  state-space  realization  of 
Eq . (4.19)  is  given  by  Theorem  4.3  below. 

Theorem  4.3 


A realization  of  the  closed-loop  optimal  compensator 
Gc  is  given  by 


where  K,  T and  Y are  given  by  Eas.  (2.9),  (3.36)  and  (3.47) 
respectively . 

Proof 

From  Chapter  3,  * 1^  has  the  realization  given  by 


where  K and  T satisfy  the  Riccati  equations  (2.9)  and 
(3.36)  respectively,  and  Y is  given  by  Eq . (3.47).  The 
realization  of  1-M  ^ * F * W is  thus  given  by 
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F 
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0 


0 

ft 

K 


! o 

- 1 

i 0 
1 


I 

t 

I 

■ t 


1 A-Kc 


ESB'V 


-A**  T 


I 


U, 


*1 


-[ 


0 


l0'Y 

’k8  Y 


+t«'T  ] 


U, 


Thus  (1-M  ^ * L«  * F * W)  ^ from  [2lJ  has  the  state-space 


representation 


A 

X 

] i i 

F ; 0 1 igg’Y  ; £gg’T 

A 

X 

8 

A 

a 

Ch  ! fi  l 0 ! 0 

i • l 

A 

a 

0 

A 

a 

0 ! K ! A-Kc  ! 0 

A 

a 

+ 

0 

— 

1 ' i 

~ 1 ■ r 1*  ' * ' - 

- - . 

6 

° : ° : e«s'y  ■ i 

i 

0 

' f '-AgsT 

1 l l 

j 

i 


X 


0 


+ 


u 


1 


(4.2  ) 


Theorem  4.3  will  follow  immediately  if  we  augment  Eq . (4.20) 
with  the  above  equation. 

O.E.D. 

Theorem  4.3  gives  a state-space  realization  of  the 
optimal  closed-loop  cascade  compensator.  The  realization 
is  explicit  and  can  be  easily  computed  from  a knowledge  of 
the  system  parameters.  Here  also,  the  compensator  equations 
contain  k as  a parameter.  The  value  of  k is  chosen  by  the 
designer  via  trial  and  error  or  graphical  techniques  such 
that  the  constraint  equation  (4.10)  is  satisfied. 

4.6  SUMMARY  AND  CONCLUSION 

In  this  chapter,  the  closed-loop  cascade  compensator 
design  problem  for  nonstationary  systems  was  formulated 
and  solved.  The  approach  used  was  a time-domain  one,  and 
the  final  design  of  the  compensator  was  given  in  terms  of 
its  state-space  realization. 

The  usefulness  of  the  closed-loop  design  versus  the 
open-loop  design  of  the  last  chapter  lies  in  the  fact  that, 
in  general,  closed-loop  systems  are  less  sensitive  to 


78 


system  disturbances  and  variations  in  system  parameters. 
But  before  such  a system  is  implemented  in  practice,  one 
has  to  be  assured  of  the  stability  of  the  loop.  This 
important  question,  that  of  the  stability  of  the  closed- 
loop  as  well  as  the  open-loop  system,  is  the  topic  of  the 
next  chapter  of  this  dissertation. 
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CHAPTER  5 
STABILITY 


5 . 1 GENERAL 

In  Chapter  5 we  derive  the  conditions  for  the  stability 
of  the  open-loop  and  the  closed-loop  compensators,  as  well 
as  the  resulting  systems. 

It  was  pointed  out  in  Chapter  3 that  the  stability 
properties  of  the  Kalman-Bucy  filter  associated  with  the 
input  signal  z(*)  play  a major  role  in  the  stability  of  the 
open-loop  system.  It  will  be  shown  in  the  sequel  that  the 
same  holds  true  for  the  closed-loop  system  as  well. 

This  chapter  is  organized  into  five  sections.  In  the 
next  section,  Section  5.2,  we  present  a brief  introduction 
to  the  stability  theory,  and  state  conditions  for  the 
stability  of  Kalman-Bucy  filters.  Sections  5.3  and  5.4 
deal,  respectively,  with  the  stability  of  the  open-loop 
and  the  closed-loop  systems.  Finally,  in  Section  5.5,  a 
brief  summary  of  our  results  and  a conclusion  are 
presented . 

5.2  STABILITY  AND  THE  KALMAN-BUCY  FILTER 

We  start  with  a few  basic  theorems  of  the  stability 
theory.  For  the  definition  of  the  terms  used  here,  the 
reader  is  referred  to  [10]  and  [24].  Consider  a linear, 
dynamic  autonomous  system 

I 

x(t)  - A(t)x(t)  (5.1) 
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Theorem  5.1  [24] 


The  zero  state  of  Eq.  (5.1)  is  asymptotically  stable 
if  there  exist  positive  numbers  and  U.2  such  that 


||4>  <t , tQ)  ||  < k1e'k2(t't0) 


for  any  tg  and  for  all  t^tg,  where  ||«||denotes  the 
Euclidean  norm.  Alternatively,  the  zero  state  of  Eq . (5.1) 
is  asymtotically  stable  if 

||4>(t,t0)||+  0 as  t->« 

Note  that  for  linear  system,  asymptotic  stability  also 
implies  exponential  stability. 

Next,  consider  the  linear  dynamic  system  E given  by 


E: 


x(t)  = A(t)x(t)  + B(t)u(t)  (5.2a) 

y(t)  = C(t)x(t)  (5.2b) 

The  bounded-input  bounded-output  (BIBO)  stability  [24] 
of  E requires  that 


/ 


|(C(t)4>(t  ,t)B(t)||  dx<k<« 


for  any  t and  all  t>tg. 

We  will  be  concerned  here  with  the  total  stability  or 
T-stability  of  systems  [24]  which  requires  that  for  any 
initial  state,  and  for  any  bounded  input,  the  output  as 

well  as  the  state  variables  of  the  system  E are  bounded. 

V 

The  following  theorem  gives  the  conditions  for  the  T- 
stability  of  E. 
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Theorem  5.2  [24. 

If  the  matrices  B and  C in  Eq . (5.2)  are  bounded  on 

h 

(“*.*)»  and  the  zero-state  of  x=Ax  is  asymptotically  stable, 
the  system  E is  T-stable. 

The  following  two  theorems  state  the  conditions  for 
the  boundedness  of  the  solutions  of  matrix  Riccati  equations, 
and  the  stability  of  Kalman-Bucy  filters. 

Theorem  5.3  [lO] 

l 

Consider  the  linear  dynamic  system  FI  given  by 

x(t)  = A(t)x(t)  + B(t)u(t) 

FI: 

y (t)  = C(t)x(t) 

and  the  matrix  Riccati  equation 

P(t)  = A ( t) P (t)  + P(t)A'Ct)  + B(t)K1(t)B'(t) 

- P(t)C'(t)K2(t)C(t)P(t)  (5.3) 

Then  if  (a)  A(t)  is  continuous  and  bounded,  (b)  C(t) , B(t), 
K^(t)  and  K2(t)  are  piecewise  continuous  and  bounded,  and 
furthermore,  that  K-^  and  K2  are  positive  definite,  (c)  the 
homogeneous  system  x=Ax  is  exponentially  stable,  the 
solution  P(t)  of  the  Riccati  equation  (5.3)  with  the  initial 
condition  P(tQ)=Pg>0  is  bounded,  and  converges  to  a non- 
negative definite  matrix  F(t)  as  tQ*-«.  F(t)  is  a solution 
Eq.  (5.3). 

Theorem  5.4  [lO] 

Consider  the  measurement  process  z(*)  of  Chapter  2 and 
its  associated  linear  model,  whose  governing  equations 
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(2.3)  and  (2.4)  are  rewritten  below 


z(t)  = yd(t)  + v(t) 

fl>(  t)  = A(t)ip(t)  + B(t)w(t) 

yd(t)  = C(t)i|i(t) 


(2.3) 

(2.4) 


Then  if  (a)  A(t)  is  continuous  and  bounded,  (b)  B(t) , C(t) , 
Q(t)  and  R(t)  are  piecewise  continuous  and  bounded,  and 
furthermore,  that  Q ( t ) and  R(t)  are  positive  definite,  and 

(c)  the  system  given  by  Eq . (2.4)  is  either 

: 

I.  both  uniformly  completely  reconstructible , and 
uniformly  completely  controllable,  or 

II.  asymptotically  stable. 

the  Kalman-Bucy  filter  given  by  Eqs.  (2.8)  and  (2.9)  is 
asymptotically  stable. 

5.3  THE  OPEN-LOOP  SYSTEM 


In  this  section  we  derive  the  conditions  for  the  T- 
stability  of  the  open-loop  compensator  and  the  overall 
system. 

Recall  from  Chapter  3 that  the  impulse  response 
Wco(t.T)  of  the  open-loop  compensator  is  given  by 


W = M"1  * L~ 
co  2 

Note  from  [24]  that  the  T-stability  of  two  or  more  tandem- 
connected  linear  systems  guarantees  the  T-stability  of  the 
resulting  overall  system.  Therefore  for  Wcq  to  be  T-stable 


it  is  sufficient  that  M~^  and  L2  be  each  T-stable. 


First 


we  consider  M 


-1 
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From  Chapter  3,  the  state  space  representation  of  M 


-1 


is 


b = (F-  i gg'T)b  + ^ g uL 
yL  = - i g’T  b + ^ uj_ 


(5.4) 


where,  T(t)  satisfies  the  matrix  Riccati  equation 

T = TF  + F’T  + h'h  - Tg  £ g’T  (5.5) 

T(t0)  = 0 

Obviously,  the  boundedness  of  T(t)  is  a necessary  condition 
for  the  T-stability  of  M ^ . The  following  theorem 
establishes  the  conditions  for  the  boundedness  of  T(t). 
Theorem  5 . 5 

Consider  the  dynamic  system  DS1  given  by  Eq.  (3.1)  and 
the  Riccati  equation  (5.5).  If  (a)  F(t)  is  continuous  and 
bounded,  (b)  g(t) , h(t) , and  k(t)  are  piecewise  continuous 
and  bounded,  (c)  k(t)  is  positive  definite,  such  that  ^ 
exists  for  all  titg,  and  (d)  the  zero-state  of  the  homo- 
geneous system  x=Fx  is  asymptotically  stable,  the  solution 
T(t)  of  the  matrix  Riccati  equation  (5.4)  is  bounded. 

Proof 

The  proof  of  Theorem  5.5  is  an  immediate  consequence  of 
Theorem  5.3. 

Next,  we  ppove  in  Theorem  5.6  that  the  conditions 
under  which  T(t)  is  bounded  are  also  sufficient  to  ensure 
the  T-stability  of  M-^.  For  the  proof  of  this  and  a 
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subsequent  theorem,  we  need  the  following  lemma. 
Lemma  5 . 1 


If  F is  a negative  definite  matrix,  the  homogeneous 
system  x=Fx  is  asymptotically  stable. 

Proof 

Choose  a positive  definite  quadratic  form 

V = x'lx 

Therefore 

V = 2x'x  = 2x'Fx 

For  negative  definite  F,  V is  negative  definite,  and 
therefore,  from  Lyapanov's  stability  criterion  [29],  x=Fx 
is  asymptotically  stable. 

Q.E.D. 

Theorem  5 . 6 

Under  the  assumptions  (a)  through  (d)  of  Theorem  5.5, 
the  system  given  by  Eq . (5.4)  is  T-stable. 


First  consider  the  homogeneous  system 


— = (F-  j7  6g'T)b 


The  state  transition  matrix  S(t,t)  of  M is  given  by 

.t 


S(t,r)  = exp[y<F-  £ gg’T)do] 


T 

* t 


“ exp 


[/f  do]  x exp  8g'T  do] 
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= $]_(£,  t)  4>2  ( c . t) 

where  <t^(t,T)  and  4> 2 (t,x)  are  the  state  transition  matrices 
associated  with  F and  - gg'T  respectively.  Denoting  the 
Euclidian  norm  by  ||  • ||,  we  have 

|^(t,T)  ||<  ||4>x  ( t , t)  ||  |B>2(t,x)  ||  (5.6) 

From  the  assumptions  and  the  consequence  of  Theorem  5.5, 
gg'T  is  bounded  and  positive  semidef inite , and  therefore 
- gg'T  is  bounded  and  negative  semidef inite . This 
implies,  from  [8]  that 

||<t2(t,  t)||  <K1<« 

Furthermore,  since  x=Fx  is  asymptotically  stable,  Theorem 
5.1  implies  that 

||<l>-^  ( t: , t)||  -*-0  as  t-** 

and  therefore 

| |S  ( t , t ) 1 1 -►0  as  t->-<i 

Theorem  5.5  also  implies  that  Jg  g and  g'T  are  bounded. 
Therefore  is  T-stable. 

Q.E.D. 

Finally,  we  proceed  to  establish  conditions  for  the 
T-stability  of  L2 . Recall  from  Eq.  (3.49)  that  the 
boundedness  of  the  nxm  matrices  and  U2  is  a necessary 
condition  for  the  T-sfability  of  L2 . Here  and  U2  are 
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solutions  of  the  first  order  linear  matrix  differential 
equations  (3.50a)  and  (3.50b)  respectively.  The  following 
Lemma  5.2  which  establishes  conditions  for  the  boundedness 
of  the  solutions  of  linear  matrix  equations  will  be 
required  in  the  subsequent  discussion. 

Lemma  5 . 2 

Consider  the  matrix  differential  equation 

X(t)  = A(t)X(t)  + X(t)B(t)  + C(t)  (5.7) 

X(tQ)  = Dq 

Here  X,  A,  B,  C and  Dq  denote,  respectively,  matrices  of 
dimensions  nxm,  nxn,  mxm,  nxm  and  nxm.  The  elements  of  A, 

B and  C are  bounded,  piecewise  continuous  time  functions, 
and  Dq  is  a matrix  of  constant  coefficients.  If  both  A and 
B are  stability  matrices  [25],  that  is,  both 


and 


Xx  = A(t)X(t) 

X2  = B'(t)X2(t) 


are  asymptotically  stable,  the  solution  X(t)  of  Eq . (5.7) 
is  bounded. 

Proof 

Let  4>^(t,x)  and  $2(t,i)  be  the  state  transition 
matrices  of  the  two  matrix  differential  systems 


and 


such  that 


A P 
Po 


1 

B 
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= A$  x ( t , t ) ; ^(t.t)  = I 


nxn 


jT  $o(t,t)  = B'$9(t,x);  $9(t,t)  = I 


inxm 


dt  - - '*’2vl"1'  • *2' 

Then  from  [26],  the  solution  X(t)  of  Eq . (5.7)  is  given  by 

t 

X(t)  = J $1(t,T)C(i)S2(T ,t)dt 

C0 

Thus  t 

llxct)!!^  ||$1(t,T)c(x)||  !|$2(i,t)l|dT 

Since  B is  a stability  matrix,  from  a theorem  in  [8] , 
4>2(x,t)  is  bounded.  That  is 

ll^Ct.x)  ||<K1<or 

and  therefore,  t 

|fx(t)  H^K1  f Kfr^Ct , t)C<t)  |(dT 
•t0 

Furthermore,  asymptotic  stability  of  P^  = AP^  implies  that 


t 

J |$1(t,T)C(-r)  ||d r <K2 <“ 


A A 


Thus  ||X(t)  ||<K^K2<°c,  and  hence  X(t)  is  bounded. 

Q.E.D. 

Using  the  results  of  the  above  lemma,  we  state  and  prove 

\ 

in  Theorem  5.7  below  the  stability  conditions  for  L2. 
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Theorem  5 . 7 


Consider  the  dynamic  systems  DS1  and  DS2  given  by  Eqs. 
(3.1)  and  (3.2)  respectively.  Under  the  assumptions  (a) 
through  (d)  of  Theorem  5.5  concerning  the  parameters  of  DS1, 
and  the  following  additional  assumptions  (e)  through  (h) 
concerning  the  parameters  of  DS2;  (e)  A(t)  is  continuous 
and  bounded,  (f)  b(t) , C(t) , Q(t)  and  R(t)  are  piecewise 
continuous  and  bounded,  (g)  Q and  R are  positive  definite, 
(h)  the  dynamic  system  DS2  is  either 

I.  both  uniformly  completely  reconstructible  and 
uniformly  completely  controllable,  or 

II.  asymptotically  stable 

the  dynamic  system  given  by  Eqs.  (3.49)  and  (3.50)  below 
is  T-stable. 


First  consider  the  homogeneous  part  of  Eq . (3.49)  given 


I 


(5.8) 


a “ (A-Kc)a 

Assumptions  (e)  through  (h)  above,  and  Theorem  5.4  imply  the 
asymptotic  stability  of  the  Kalman-Bucy  filter  associated 
with  the  system  DS2.  Consequently,  the  homogeneous  system 
of  Eq.  (5.8)  is  asymptotically  stable,  and  futhermore,  K is 
bounded . 

Next,  we  prove  that  under  the  assumptions  of  the 
theorem,  U^+U2  is  bounded.  In  Eq . (3.50a)  for  , F'  and 
(A-Kc)  are  stability  matrices.  Therefore,  from  Lemma  5.2, 
is  bounded.  Furthermore,  in  Eq.  (3.50b)  for  U2 , 

F'-J^  Tgg'  is  also  a stability  matrix  from  Theorem  5.6. 

Thus  U2  is  also  bounded.  Hence  U^+U2  is  bounded  and  the 
T-stability  of  L 2 follows. 

Q.E.D. 

The  above  discussion  establishes  the  conditions  for  the 
T-stability  of  the  open-loop  compensator  W =M~^*L2. 

Since  the  impulse  response  of  the  overall  system  is  W*Wco , 
and  since  W must  be  T-stable  in  accordance  with  the 
conditions  of  Theorem  5.6,  the  overall  system  is  T-stable. 

In  summary,  we  thus  conclude  that  the  conditions  (a) 
through  (d)  of  Theorem  (5.6),  and  the  conditions  (e) 
through  (h)  of  Theorem  (5.7)  guarantee  the  T-stability  of 
the  open-loop  compensator,  and  that  of  the  overall  system. 
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5. A THE  CLOSED-LOOP  SYSTEM 

In  this  section  we  discuss  the  stability  properties  of 
the  closed-loop  compensator  and  the  closed-loop  system  of 
Chapter  A.  We  show  below  that  the  stability  of  the  closed- 
loop  system  (but  not  necessarily  that  of  the  closed-loop 
compensator)  is  a direct  consequence  of  the  stability  of 
M"1  and  L2. 

From  Chapter  A,  we  have 

y (t)  - W * Gc  *{ 1+F  * W * Gc_1  } z(t)  (A. 12) 

where  Gc , the  impulse  response  of  the  closed-loop 
compensator  is  given  by 

Gc  = U-M-1  * L2  * F * W]-1  * M'1  * L2  (A.  19) 

Substituting  Eq . (A. 19)  into  Eq.  (A. 12),  and  for 
convenience,  omitting  the  asterisk,  we  obtain 

y (t)  = W(l-M"1L2FW)_1M'1L2{l+FW(l-M'1L2FW)'1M"1L2)1z(t) 

(5.9) 

Now  consider  the  bracketed  portion  of  Eq.  (5.9),  which  we 
rewrite  below. 

{l+FW(l-M"1L2FW)'1M_1L2r1  (5.10) 

Expanding  (1-M”^L2FW)  * into  a Neumann  series  [28]  Eq . (5.10) 
is  written  as 
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{ 1+FW ( 1 -M" XL2  FW+M‘ XL2  FWlf lL2 FW- ) M" 1 L2 } " 1 

- {l+FWM'1L2-FWM"1L2FWM'1L2+FWM"lL2Fl^M'1L2nW"1L2.  . . J"1 

- {(l-FWM"1^)'1}'1 

= i-fwm_1l2 

Thus  Eq.  (5.9)  reduces  to 

y (t)  = W(l-M'1L2FW)_1M'1L2(l-FVJM'1L2)z(t)  (5.11) 

By  expanding  (1-M  ^L2FW)  ^ once  again  into  a Neuman  series, 
it  is  straightforward  to  show  that 

(1-M"1L2FW)'1M'1L2  = M‘1L2(1-M"1L2FW)"1  (5.12) 

Substituting  Eq.  (5.12)  into  Eq.  (5.11),  we  obtain 

y (t)  = WM‘1L2(l-M"1L2nO”1(l-M'1L2FW)z(t) 

- W * M-1  * L2  * z ( t)  (5.13) 

The  discussion  so  far,  on  the  closed-loop  system  leads  to 
the  following  theorem. 

Theorem  5 . 8 

Under  the  assumptions  (a)  through  (d)  of  Theorem  5.5 
concerning  DS1,  and  the  assumption  (e)  through  (h)  of 
Theorem  5.6  concerning  DS2,  the  closed-loop  system  of 
Fig.  (4.1)  is  BIBO  stable. 

Proof 

The  proof  follows  immediately  from  Theorems  5.6,  5.7 
and  Eq.  (5.13). 

Q.E.D. 
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It  should  be  noted  that  Theorem  5.8  gives  conditions 
for  the  BIBO  stability  of  the  closed-loop  system.  The 
T-stability  of  the  system,  in  addition  requires  the  T- 
stability  of  the  closed-loop  compensator  and  the  feedback 
sensor  (DS3)  with  impulse  response  functions  and  F 
respectively.  From  Eq . (4.19) 

Gc  = [ 1-M-1  * L2  * F * w]'1  * M_1  * L2 

Thus  in  addition  to  the  conditions  of  Theorem  5.8  which 
imply  the  T-stability  of  M-^  * L2 , the  T-stability  of  the 
closed-loop  system  requires  the  T-stability  of  the  operator 
[l-M~*  * L2  * F * w]"^.  The  additional  conditions  which 
must  be  imposed  on  DS1 , DS2  and  DS3  for  the  T-stability  of 
[l-M  ^ * L2  * F * w]  1 are  left  as  a topic  of  future 
research  on  this  problem. 

5.5  SUMMARY  AND  CONCLUSION 

In  this  chapter  we  derived  conditions  for  the  T- 
stability  of  the  open-loop  compensator  and  the  open-loop 
system,  and  the  BIBO  stability  of  the  closed-loop  system. 

It  was  shown  that  the  conditions  which  must  be  imposed  upon 
the  plant  (DS1)  and  the  reference  system  (DS2)  for  the  T- 
stability  of  the  open-loop  system,  also  guarantee  the  BIBO 
stability  of  the  closed-loop  system.  The  T-stability  of 
the  closed-loop  system  requires,  in  addition,  the  T- 
stability  of  the  closed-loop  compensator.  The  T-stability 
properties  of  the  closed-loop  compensator  are  left  as  a 
topic  of  future  research  in  this  area. 
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CHAPTER  6 


CONCLUSIONS  AND  RECOMMENDATIONS 
FOR  FURTHER  RESEARCH 

6.1  CONCLUSIONS 

The  object  of  this  research  was  to  develop  a unified 
theory  of  optimal  tracking  (in  the  mean-square  sense)  which 
would  admit  stationary  as  well  as  nonstationary  systems. 
Only  single-input/single-output,  multi-state  systems  were 
considered  here.  The  research,  in  essence  involved  the 
design  of  compensators  which  would  give  the  systems  under 
study  the  desired  tracking  properties. 

The  systems  in  both,  an  open-loop  and  a closed-loop 
configuration,  were  studied.  For  each  of  the  configura- 
tions, the  optimal  compensators  were  realized  explicitly 
in  the  state-space.  The  compensator  equations  contain  an 
unknown  parameter  (a  Lagrange  multiplier) , whose  value  is 
chosen,  usually  by  graphical  techniques,  to  limit  the  input 
signal  to  the  plant. 

The  question  of  the  system  stability  for  the  two 
configurations  was  also  addressed.  It  was  shown  that  if 
the  Kalman-Bucy  filter  associated  with  the  input  signal, 
and  the  given  plant  are  asymptotically  stable,  the  open- 
loop  system  is  totally  stable  (T-stable) . It  was  also 
shown  that  for  the  same  conditions,  the  closed-loop  system 
is  only  bounded-input  bounded-output  stable.  Additional 
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conditions  which  must  be  imposed  upon  the  closed- loop 
system  for  the  T-stability,  need  further  research. 

6.2  RECOMMENDATIONS  FOR  FURTHER  RESEARCH 

The  theory  of  stochastic  optimal  tracking  of  nonsta- 
tionary systems  is  by  no  means  complete  at  this  time. 

Some  related  problems  and  areas  which  warrant  further 
research  are  given  below. 

1.  Conditions  for  only  the  bounded-input/bounded  out- 
put stability  of  the  closed-loop  system  are  given 
here.  The  question  of  the  T-stability  of  the 
closed-loop  system  warrants  further  research. 

2.  Recall  from  Chapter  5 that  a necessary  condition 
for  the  T-stability  of  the  open-loop  system,  and 
the  BIBO  stability  of  the  closed-loop  system,  is 
the  asymptotic  stability  of  the  plant.  What  if 
the  plant  is  unstable  to  start  with.  The  open- 
loop  compensator  is  of  no  direct  value  here,  unless 
the  plant  is  stabilized  first  by  an  auxilliary 
closed-loop  [2].  A possible  alternative  is  the 
closed-loop  compensation  scheme  along  the  lines  of 
Youla,  Jabr  and  Bongiorno  [5]  for  the  stationary 
case.  Thus  if  the  unstable  plant-sensor  combina- 
tion can  be  stabilized  at  all,  and  if  Z is  the  set 
of  all  possible  stable  and  proper  compensators 
which  stabilize  the  system,  then  the  search  for  the 
optimal  compensator  need  to  be  restricted  to  this 
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set.  An  extention  of  this  method  to  the  case  of 
nonstationary  systems  is  nontrivial  and  warrants 
considerable  research. 

3.  The  case  of  multi- input/multi-output  system  and 
colored  noise  should  be  investigated.  Extention 
of  all  of  the  above  work,  completed  or  suggested, 
for  the  discrete  time  systems  is  of  great 
practical  value. 
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